Tips to Incrementally Develop a Robust Test Automation Framework

Introduction

Test Automation Framework is creating set of methods to test software application under certain rules and to execute efficiently the tests on software application without manual intervention. There are different types of test automation frameworks based on their capability to test. The simplest form of test automation framework that can be as simple as record and playback but as the complexity of the software to be tested increases with increasing frequency of testing, and also when framework required to test different configurations, resolution and requirements across different environments, we need more robust test automation framework.

As there is no one type of test automation solution that can fit all situations to test. The basics rules of test engineering process remains same but test automation requires to be developed keeping in mind the functionality or behaviour to be tested for the application under test. As software requirements is different for different projects, the test automation solution also required to test different projects cannot be generic.

The primary goal of test automation framework is to reduce manual intervention and reduce cost of testing. Although the initial investment for developing the framework from scratch is high it is beneficial for long term, as tests once developed can be used over again and again with minimum maintenance and high reusability and efficiency.

Types of Test Automation Frameworks

There are primarily five types of test automation frameworks

• Record & Playback (Script is recorded once and played back multiple times)
• Data Driven (Input Data feeds the tests through excel or database and different combination of input determines different tests)
• Keyword Driven (Input action word determines the test to be performed)
• Modular Framework (Tests are maintained separately by maintaining methods as segregating functionality wise)
• Hybrid Framework (Tests maintained by combination of two or more above methods)

Planning the Test Automation Framework

To develop the robust test framework one should start with understanding the application to be tested first, without jumping to creating the test automation framework.

• Determine the common function that can be automated
• Prioritize of the features to be tested by analysing the application based on importance of features
• Based on behaviour of application select the appropriate type of test automation framework and if you are planning for hybrid or custom framework determine which data structure or design pattern will best handle the complexity of application and then the test framework becomes maintainable.
• Analyse different aspects and non standard parts of application like custom controls, CAPTCHA controls, video streaming, etc those which cannot be automated and classify as not automatable.
• Plan incremental approach for framework development rather than big bang approach

Create Prototype of the working Test Automation Framework

• Create basic modules of framework based on common functionality and priority of feature to be tested
• Device a layered approach by segregating modules in different layers for example, Test Case Layer at to level, Module Layer for domain functionalities, System Level Layer for software platform and OS functionalities, Common Layer for generic functionalities and Foundation Layer for framework specific methods.
• Insteading reinventing the wheel, wherever possible make use of utilities, automation tools that are proven for the purpose which could reduce the time of development of framework. For example reporting module can make use of readily available test reporting plugins, making use of  existing xUnit, unit testing frameworks for categorization and execution strategy.  
• Make use of best practices and coding guidelines

Mature the Test Automation Framework

• Automate a small set of test cases and note down the shortcomings or gaps and incrementally fill those gaps.
• Modify framework to be adaptable with multiple environment or configuration
• Enhance the framework to scale up to handle varied types of test suits for different purposes like Smoke, Sanity, Integration, Regression, End to End etc...
• Be flexible enough to modify the framework to include the enhancements as framework should in evolution mode most of the time. Give dedicated time for framework enhancement.
• Add logging module to track execution and troubleshooting
• Integrate automation framework with Continuous Integration tools
• Incorporate best practices in test engineering and test automation processes
• Keep the code clean and up to date. If possible make use of code analysis tools for analysing automation code.
• Document important parts of automation framework.
• Have a plan and minimize false positive, false negative, for such resultant tests to find the root cause and fix them. 

Conclusion

Instead of big bang approach to develop test automation framework, if we follow incremental and step by step process by using standard practices in testing, best practices and proven utilities for test automation the robust test automation framework so developed thrives.