Test Management encompasses anything and everything that we do as testers. Our day-to-day activities include:
- Creating and maintaining release/project cycle/component information
- Creating and maintaining the test artifacts specific to each release/cycle that we have- requirements, test cases, etc.
- Establishing traceability and coverage between the test assets
- Test execution support – test suite creation, test execution status capture, etc.
- Metric collection/report-graph generation for analysis
- Bug tracking/defect management
The above are broadly some of the tasks that involve what we call, the test management process. This process is critical, detail-oriented and instrumental is making sure that the entire testing effort is successful.
Well, the good news is, there is help available. Through this article we will try to introduce ourselves to the most commonly available tools for the test management process briefly. Here is the comparison of top test management software.
List of the Most Prominent Test Management Tools
QMetry is a total test management tool that you can use to create requirements, test cases (test suites) that can be run on multiple platforms and defects. It integrates seamlessly with many leading bug-tracking and automation environments making a good candidate for use in most situations. It is a commercial product, with a 30 day free trial available. Visit the site for more information at:http://www.qmetry.com/
TestRail is a centralized test case management tool – you can use it to create test cases and test suites, track execution and report metrics. Additionally, it integrates with many issue tracking tools that makes requirements from external systems to be linked to test cases in TestRail; bugs can also be created in the external systems and links can be established to the corresponding test case. It comes with a HTTP-based API to integrate with the automated test results. One most common integration is with Gemini, which is a incident/ticket management system(supports agile too). It is a commercial product with a free trial available at:http://www.gurock.com/testrail/
JIRA is tool that makes an appearance anytime there is a discussion on any-management process- for all the right reasons. JIRA has 2 add-ons that support the test management process.
a) Zephyr: All the aspects that you would expect of a typical tool of this type are supported. You can create tests/test suites/test cycles/bugs/reports and so on. You could have an additional add-on, ZAPI for automation integration. Along with the initial JIRA license you would have to pay for Zephyr to use it. ($10 for 10 users a month). There is a free trail available too. Check out information about it at:https://marketplace.atlassian.com/plugins/com.thed.zephyr.je
b) Go2Group SynapseRT: This tool has all the test management features but the primary focus is on requirement based testing. It can be used for projects where it makes more sense to track your progress in terms of the completion and/or success-failure of a certain requirement as opposed to test cases. Traceability is of higher priority with this add-on. Along with the initial JIRA license you would have to pay for this add-on too. ($10 for 10 users a month). There is a free trail available at:https://marketplace.atlassian.com/plugins/com.go2group.jira.plugin.synapse
Developed by QASymphony, qTest is one of the cloud-based test management tools that has all the typical key features. With the help of qTest Connector, it can integrate with JIRA for an entire end-to-end QA solution – but that is not all, it also integrates with other tools like Bugzilla, FogBugz, Rally etc. It is not open sourced but very affordable. Check out information and pricing at:http://www.qasymphony.com/qtest.html
This is a comprehensive test case management tool that has 4 key aspects – Test Plan, requirements, Test suites/cases and test runs. So, as you can see, it has everything it needs to be able to manage test cases for you. For all the other operations, it integrates with the many main stream incident/issue management tools to provide a comprehensive solution. It is a commercial product, for a free trial visit: http://www.testlodge.com/
6) HP ALM/Quality center:
HP QC has been one of the most used test management softwarefor many years. It has all the features necessary and in many ways, it is the standard against which the other tools are measured. Even though it is one of the high-end tools, economically, it still remains to be very popular. Check out http://www8.hp.com/in/en/software-solutions/software.html?compURI=1172141 for trial and other information.
This is the same Zephyr that we discussed earlier as being an add-on to JIRA. The reason why it deserves an independent discussion is because it can be used as an individual product also. For free trial and more info: http://www.getzephyr.com/
Testuff is Saas Test Management tool that has many cool features. The typical test case management features are a given. Besides that, it has a cool video upload capability for a defect. Integrates with a huge list of bug-trackers, some of which are, Bugzilla, JIRA, YouTrack, Mantis among others. It has an API that supports automation tools like QTP, Rational Robot, Selenium, TestComplete etc. Best of all, it is very affordable. Check out the features and pricing information at:http://www.testuff.com/
9) Test Collab:
A web based test case management tool that states its speed to be one of its key features due to its ‘Ajax’ified interface(in the exact words of the software makers). It is simple to use, integrates with all major bug/issue/incident trackers. Customizable and has a good reporting facility. It is a commercial product and information is available at: http://testcollab.com/
One of the key components of this tool is supporting ‘Testing & QA’ along with the other aspects like Project Planning, issue tracking etc. Using this tool you could create test plans, test cases, test runs, traceability, test run reports etc. There are also various integrations and extensions available. It is a commercial product with a free starter pack available at:http://www.countersoft.com/solutions/testing/
It is an end-to-end QA and Test management software. You can organize your requirements, create tests, run tests, tracks bugs etc, using this tool. It integrates very well with three of the leading incident management tools like JIRA, Bugzilla and RedMine. It is not open sourced but is quite affordable. For more feature and pricing information, check: http://www.practitest.com/
This is one of the very few open source test management toolsavailable for use in the market. It is a web- based tool with typical features like, requirement management, test case creation and maintenance, test runs, tracking bugs, reports, integration with common issue trackers etc. To download, visit:http://sourceforge.net/projects/testlink/
QAComplete is one of the most powerful test management tools that we have. This suits agile/traditional, manual/automation projects excellently. You could use it in integration to QTP and TestComplete. For automation projects, you could schedule the test runs and run them remotely on any registered hosts. There is also a detailed release management feature that provides for better analysis. It works along with major bug-trackers and with source control tools – Subversion, Perforce and CVS. Given all the features, it is a little pricey. There is a free trial available though. Check out all its features at: http://smartbear.com/products/qa-tools/test-management/
14) Silk Central:
This is a test management product by Borland. Once you have Silk Central with you there is a nothing left for you to ask for anything else. It is robustness personified, though it is pricey. All the features for coverage, traceability, reporting, test creation, running – it has got them. It integrates with many source control and issue tracking system. There are additional plug-ins to extend its capabilities to automation testing using QTP, WinRunner etc. It comes with a video capture feature and supports SAP testing. This product is really cool. Try it at: http://www.borland.com/products/silkcentral/
15) IBM Rational Quality Manager:
A test Management product that has all the typical features – Test planning, test design, test execution, tracking and reporting. It integrates with many of the rational products for automation, source control and bug tracking activities. It is a commercial product. Check out its features, pricing and other information at: http://www-03.ibm.com/software/products/en/ratiqualmana
The below are some more tools that are worth mentioning:
16) VersionOne: A commercial product primarily catering to agile projects, this has a test management module along with planning, reporting and others, with all the typical features. Check out:http://www.versionone.com/product/agile-test-management/
17) TestPad: This tool’s motto is “Spend more time actually testing”. The primary concept of this tool is – checklists. In your test plan you could have a series of checklists (tests) that can be as detailed or as short as possible. It is perfect for exploratory testing. It is commercial and you can try it at: https://ontestpad.com/
18) Aptest: a web based, commercial product that has all the typical features you would expect along with the typical integrations with issue trackers. http://www.aptest.com/atm2/
19) SpiraTest: A complete QA solution is what this product is. Instead of having your requirements, tests, defects in separate systems, this tool has it all in one place. It integrates with unit and automation testing frameworks among other things. It is commercial, though not very expensive.https://www.inflectra.com/SpiraTest/Default.aspx
20) Meliora TestLab: This is a simple to use product for requirement management, test-creating & running, defect management, workflow optimization, and integration with JIRA etc. Commercial again, check it out at: https://www.melioratestlab.com/
21) SmarteQM: A complete life cycle management tool that provides the complete end-to-end test process support and integrates with other SmartSoft functional test tools. It is web based and commercial. http://www.smartesoft.com/products_smarteQM.php
22) Test Run: a web based commercial test management process that is easy and simple to use. It has all that you need to create test plans, execute them successfully and report. Integrates with JIRA and LightHouse. http://runtestrun.com/
23) Test Wave: A test management tool that needs no installation, is web based, simple and commercial (affordable). This tool provides you a facility to import your already existing requirements/test assets from excel sheets. It also comes with an inbuilt defect trackerhttp://www.testwave.co.uk/
24) Enterprise Tester: This test management tool supports both agile and traditional projects. Integrates with JIRA and also works great for automation testing with QTP, Selenium, RFT etc. This is a really cool commercial tool. Check it out at:http://enterprisetester.com/
25) TestLog: A very comprehensive end-to-end test management tool that is easy to install and configure due to its XML database. It allows documentation of both automation and manual test cases. It also comes with a web interface for remote access. This product is commercial too. http://www.testlog.com/
26) QaTraq: An open source test process control tool that can be used to create test cases, running them, recording results etc.http://sourceforge.net/projects/qatraq/
Points to note:
Well, from the above list two things are apparent:
- There are not many open source tools of this type available, although most of them are still affordable.
- Most of the tools just provide test case management and leave the bug tracking to be integrated via an external tool. (A smart choice, if you ask me. Otherwise, all of us would be stuck reinventing the wheel.)
Even though we put up a big list, we realize that we could not have included every tool available. Share your experiences with the test management tools – both the ones that made it to the list and those which did not- with us below.