The Extent to automate - The real value to unleash

Introduction

In large software development projects, one of the key metrics for success of automation lies in the extent to which we can automate. Automation generally makes more of the ROI from the amount of the time saved by automating the test cases which save considerable amount of manual effort but the  development and maintenance cost involved is often overlooked.  Today, we discuss the extent to automate while developing cost effective automation frameworks.

Framework Design - Make the most out of re-usability and Modualarity

Framework design should take care of high degree of cohesion and  low coupling.  Low coupling enables relationship between modules to be simple and stable interface and does not need to be bother about other module's internal implementation. The framework design should also incorporate re-usability of code and data within the test framework. The kind of model to be used viz. Page Object Model, Keyword Driven Model, Data Driven Model or Custom model which can be combination of any of these models can be has to be decided based on behavior or the type of application under test.

Code and Data Generators - The real value in automation

Often the frameworks under development have a certain pattern to be followed. For example while using the Page Object Model implementation a class structure with types representing controls needs to be created, as the syntax of programming language is known and the type of controls on respective page by using Regular Expression, code can be generated for those classes automatically by making separate utility or by incorporating it in framework. Thereby reducing lot of effort creating code which is repetitive. Similarly, like code generators depending on type of data to be generated, data generators can be developed, by knowing test case pattern and expected result of the application under test.

Defining the scope of Automation - Tracking the artifacts

Even though 100% automation is not possible, It is utterly important to classify the test cases as automatable, not-automatable and critical. Attributing the test cases with test case number and / or requirement number and additional information and capturing it as part of Test Automation Reports and analyzing them with reporting tools can present meaningful information on overall planning and execution of Test Effectiveness.

Capturing and analyzing logs - Intelligent logging mechanism

Following a certain pattern in logging mechanism while writing the logs and developing support utilities to analyse these logs, this would enable to detect false positive test result or data set used for particular failed test case etc.  and implementing logic in test framework to appropriately take action like re-executing the test cases or using a different data-set for a particular test case and re-executing a failed test case in case to test if it passes in second run.

Conclusion

To conclude, test automation can be a valuable asset, when implemented with robust framework design along with code and data generation capabilities, which significantly reduces the framework development effort. Capturing additional information can enable comprehensive reports and intelligent logs which is capable of deriving meaningful information and take appropriate action and help reporting mechanism thereby adding value to the automation and reducing the maintenance efforts.