Analysing Test Automation Reports
Introduction
Like any other reports, automation reports reflects the the state of application under test. Along with the status we can draw meaningful insights from automation reports which can be a boon to test automation efforts. Almost every test automation gives status like pass, fail, errors etc. Today we will try to strategies to analyse the test automation reports.
Why Analysis?
With the status of test automation report one can take appropriate action, on failure we can check what is the cause of failure and take appropriate action. For example if the test is failing due to bug in the application then raise a bug. If the failure is due to bug in test automation framework then fix the test automation framework. What is essential is to perform analysis without which test automation report which is automatically generated cannot by itself provide confidence to results of status of the test. Additional quality tools can be applied and extract meaningful information from the defects and help broad decision making.
How To Perform Analysis?
- Setup the process to collect the test report automatically.
- Analyse each of the failure to know the cause of the failure and classify as Bug in Application, Bug in Test Framework or Environment issue
- Collect as much data as possible from reports and bugs found during the test execution
- Data so collected during analysis can be used to draw meaningful insights in the test automation reports.
- Make use of quality tools to perform the analysis
Tools To Perform Analysis
- Histogram
- Check Sheet
- Cause and Effect diagram
- Pareto Charts
- Scatter Diagram
- Control Charts
- Stratification (Flow Chart)
Conclusion
Post a Comment