IP-DRIVEN

TEST AUTOMATION FRAMEWORK Accelerating Product Launches/A Journey in Innovation

ABSTRACT In this supersonic age, when IT products and solutions are churned around at breakneck speed, quality sometimes is sacrificed at the altar of time. But, the consumer is more aware and more demanding, and any flaws or bugs can cost the company its reputation and business. Quality assurance needs to keep pace and automation has helped speed up time to market. A QA automation framework ensures speed while also providing for customisation to suit each product being tested with minimum time to set up test cases. An IP-driven testing framework is not just a product but an approach that is flexible, agile and evolving, keeping pace with and anticipating technological developments.

© Indium Software | www.indiumsoft.com

TEST AUTOMATION – MARKET OVERVIEW Software solutions are becoming a critical and integral part of businesses on the one hand, and sometimes, the key driver too, as in the case of e-commerce and banking. Therefore, developers are under tremendous pressure to design and execute solutions and products that operate smoothly, cover all critical aspects of doing business, ensure security and are scalable. This has boosted the testing industry, which works hand in hand with software development, ensuring the solution achieves its goal and assuring quality. The demand is further fuelled by the spurt in mobile applications and cloud technology. Over the years, testing has evolved from being manual to integrating test automation for speed and accuracy. A Transparency Market

Research report, “Test Automation Market - Global Industry Analysis, Size, Share, Growth, Trends and Forecast 2016 - 2024,” indicates that the global test automation market, valued at US$ 12.91 Bn in 2015, will grow at a CAGR of 23.6 percent between 2016 and 2024, when it will reach US$ 85.84 Bn. Test automation includes functional testing, configuration testing, web services testing, acceptance testing, compatibility testing, integration testing, load testing, security testing, mobile testing, migration testing, platform testing, usability testing, network testing, and QA process design. Functional testing contributed the highest at 23.2 percent, and is expected to maintain its lead, while security testing is expected to be the fastest-growing.

TEST AUTOMATION TOOLS – EVOLUTION Testing as a process requires several steps to be repeated. So, it was natural that these repetitive actions were automated to reduce the burden on the testers themselves, who could then focus on unique aspects that require test cases to be written afresh every time. The market started seeing tools that helped testers automate the process.

© Indium Software | www.indiumsoft.com

From Rigid Ones…

The first generation test automation tools were primarily record and playback, especially useful for regression testing and could run unattended for a few hours. Every test case was recorded as a series of actions with test data hard-coded into it and any small change in the application required the test case to be re-recorded, thereby making it complex to maintain the automated scripts. Needless to say, this had severe limitations as it needed manual intervention for every change and required tools to catch up to the quick developments in software development technology. Testers realised that the tools were limiting instead of becoming true enablers and required coding skills. Every time they had to modify or write new scripts and dump the old ones, thus putting in double the effort to achieve the same result.

Limited Impact

Earlier, products had a lifecycle, and each version would be an upgrade till the old one was eased out to be replaced by a new one. But with time, products take completely new avatars and testing also needs to be able to cope with these changes. The test automation tools could only be modified after the changes in applications were introduced, and became irrelevant quickly after the next - unproductive like running a marathon standing in one place; and a drain on resources and investments. And to top it, such tools enable only 30-40 per cent automation of testing due to the need for constant manual monitoring and intervention.

Being Flexible

Indium, realising the limitation of test automation tools, opted to develop a framework, culled from its vast experience in testing products catering to a cross section

of industries. Most applications have fundamentally the same structure and only features are added. Thus, there are many reusable features that do not require new scripts to be written every time, and reusable test cases are stored in a library that can recalled as and when needed. It is key/action driven, and does not need testers to know coding, making it easy for them to write scripts only for new features. It also has in-built intelligence and helps testers understand how much automation is needed for an app. The application upgradation frequency is also increasing and as against a new build every two to three months, today sometimes testers have to handle 7-8 builds per day. Each build has to be compared and the differenced capture for comprehensive testing. Indium’s i-Accelerate IP-driven framework is intuitive and flexible, thus ensuring setting up of automated testing quickly and for accurate results.

© Indium Software | www.indiumsoft.com

EVOLUTION OF TEST AUTOMATION Modular Second generation

Data driven Third generation

Record & playback First generation on

Keyword & hybrid Fourth generation

JUMP START TESTING Across industrial segments, developers have certain common underlying requirements that have become the foundation blocks for test automation framework: • Faster time to market • Constant additional enhancements • Reduced regression run time • Enhanced test coverage

PROPRIETARY TOOLS Proprietary test accelerators plug into every stage of the SDLC to bring in the best-in-class enablers for reducing the TCO and delivering tangible RoI. Open-source/licensed solutions that are tool agnostic ideally meet this requirement. © Indium Software | www.indiumsoft.com

KEY DIFFERENTIATORS IP-led testing frameworks are: • Customizable • Portable • Re-usable • Maintainable • Assure quick ROI on test automation They are capable of enhancing test coverage, running target based parallel execution and efficiently analyse the root-cause.

THE SOLUTION APPROACH Applications today are not only accessed on a PC, but are webready too, due to the proliferation of the Internet technology in all aspects of doing business. It poses its own set of challenges, needing the solution to be OS and browseragnostic.

WEB-READY (iSAFE)

An efficient tool can run compatibility tests in parallel, thus reducing test time, continue uninterrupted even when encountering failures, depend less on technically skilled resources, have a dashboard designed for reporting test results that are contextual and enable quick decision making. Perform database and API test automation further ensure a better quality software. An enhanced dashboard, built-in-analytics and report generation tools will help product owners improve the development process and overcome common challenges. Additionally, User Interface testing will also be incorporated in future versions.

INDIUM’S IP-DRIVEN IACCELERATE TOOLS

iSAFE - Indiums’ Software Automation Framework Enhanced

iMobi - Indium’s Mobile Automation Framework

Framework to address web testing requirements needs to be cost and time efficient and also differentiate between critical and non-critical bugs. It needs to be browser and OS-agnostic, be customizable and portable and the test scripts should be highly reusable.

© Indium Software | www.indiumsoft.com

iFACT - Indium’s Framework for Accelerated Compatibility Testing

INDIUM’S IP-DRIVEN IACCELERATE TOOLS • Filters, sorting and pagination • Hyperlinks and broken link validations • Export to excel validations • Verification of texts in elements and page • Verification of page titles • Email content verifications and attachment validations • Graph validations and data comparisons

Mobile Apps

Mobile is the next important development that has caught the development world by storm. It has made access to services easier, even when on the move, for consumers, who need secure and faster services delivered on their mobile devices.

• Alert validations and verifications • Handling of dynamically created data by the previous tests • Verifications required for checkboxes and radio buttons • Checking attributes of web elements • Dropdown sections related • Verification of visibility, enabled

and clickability of elements. • API Request and response validations in JSON and xml formats • DB Queries and validation of resultsets • Batch monitoring and log validations

A dedicated mobile lab that has several hundred mobile devices including of iOS, Android and Windows devices for compatibility testing is a must to ensure deviceagnostic testing. Devices come in different screen sizes, and each device type has its own unique set of compliance requirements. Regardless of whether the mobile

app is Native, Hybrid or Web, Functionality, Compatibility and Performance are critical to app testing on different devices. QA on the cloud and a user interface are some of the additions planned as part of the continuous improvement process.

KEY VALIDATIONS FOR MOBILE READINESS • Run across multiple devices in parallel, reducing test life cycle time • Jump-start testing and save 2-4 weeks of time • Needs minimal scripting

• • • •

knowledge It is test tool agnostic Leverages best practices Reporting over mail or SMS after every test Cloud ready, enabling QA on

the cloud • Interfaces with multiple tools • Hybrid automation framework • Is compliant with all UI/UX guidelines for the various OS

© Indium Software | www.indiumsoft.com

iFACT – for Compatibility Testing

With the Internet becoming critical to businesses, firms need to ensure that their web applications are compatible with the environments of their target audience – the web applications need to be comprehensively tested in compatibility to browsers and OSs. iFACT (Indium’s Framework for Accelerated Compatibility Testing) was developed to help firms eliminated compatibility issues and gain competitive edge.

iFACT ENABLES • Testing of applications across all combinations of internet browsers and desktop OS • Reusing test scripts and minimizing time to market • Leveraging Selenium (open source automation tool) for automating a significant portion of the test framework

© Indium Software | www.indiumsoft.com

FOUR STEPS TO SUCCESS A successful implementation of quality assurance project is ensured by a methodical approach that consists of the following four steps:

FLOW CHART Defining the scope of Test Automation

Understanding of Application/Test cases

Identification of Reusable Components

Sharing the observations/gaps in test cases

PLANNING

Creation of Reusable Components, Objects Repository (OR) Designs

Creation of Test Case & Test Data Designs

Maintenance of clarification tracker

Checkpoint meetings

DESIGN

Customization of Framework & integration

Implementation of data driven approach

Development of Test scripts

Checkpoint meetings

IMPLEMENTATION

Execution of Test Scripts Unit/Batch

Stabilization of Test Scripts

Exception handling/ Recovery scenarios

Capture failures and real-time reporting

EXECUTION

© Indium Software | www.indiumsoft.com

PLUS ONE – COMMUNICATION Constant interaction with customers over multiple tiers of communication channels, real time bug fixing, suggestions on design improvement and monetisation are some of the other key value added services are some of the additional roles of an efficient QA partner.

CASE STUDIES The proof of the pudding is in the eating. Our testing teams have several successful testing automation instances that confirm the efficiency and the effectiveness of our framework and the suite of tools. For one of our Canadian customers – which provides solutions for schools and educational institutions, we ran compatibility tests across multiple operating systems, devices and browsers. Additionally, we also executed tests on cloud based services. A Florida based company, which also works with higher educational institutions, was dissatisfied with TestNG reports and the other readymade reports. We provided friendly and customizable reports generated using Indium’s reporting engine. The reports were dispatched at the end of test execution to all the stakeholders configured in the framework, thus cutting down on communication times.

© Indium Software | www.indiumsoft.com

RESEARCH & DEVELOPMENT Dedicated research and development team is critical for continuous improvement to keep pace with the fast paced technological development. The team not only does its own research, but also works closely with implementation teams when encountering any difficulties and integrates the learnings in the framework, thus constantly improving the product.

CONTINUOUS IMPROVEMENT Using Excel for benchmark information, UI for making changes in test cases even by non-programmers are some of the rollouts to improve the framework. Having a roadmap enables focused improvement of the framework and plan budget projections.

SHARING IP For longstanding customers, free access to the source code of the IP-driven testing automation framework as a reward for loyalty is provided along with training and subsequent trouble shooting. Customers can also continue the relationship and benefit from future updates too. This is a unique offering that has been well appreciated. © Indium Software | www.indiumsoft.com

INDIUM SOFTWARE – OVERVIEW Established in 1999, Indium Software is a global Independent Software Testing Services Company with multi-domain focus, fostered by IP-led innovation. The core of Indium’s objective to servicing our global customers can be explained with this simple line: “We’re small enough to care, large enough to deliver.” Indium has offices in California, New Jersey & Georgia, USA; Kuala Lumpur, Malaysia and Global Delivery Centers in Chennai & Bengaluru, India. Our 450+ career testers work with a mix of enterprise and ISV clients ranging from fortune 100 to 5000 companies in the US, Canada, India, Malaysia, Singapore, Middle East, UK, Netherlands and Turkey.” Critical to Indium’s strategy for accelerated testing is its frameworks, which are branded under ‘iAccelerate’ suite of frameworks

& tools. It is built based on best practices delivered over many years to clients globally. Indium has strong competency in Banking, Technology, Healthcare, Lifesciences, Retail, Education and Gaming industries. Indium is aggressively pursuing the social, mobile and cloud agenda to position strongly for the emerging paradigms. Testing is a critical part of the work we do. But that is just one part. At Indium, being a partner, understanding our customers’ needs and providing solutions that can help them roll out their qualityassured product earlier forms the bulwark on which the framework is developed and run. Continuous improvement as part of our endeavour to achieve this gaol, and innovate for optimum results, our strategy.

© Indium Software | www.indiumsoft.com

CONTACT US USA SUNNYVALE Suite 210, 1250 Oakmead Parkway Sunnyvale, CA – 94085. Phone: +1(408) 501-8844 Fax: +1(408) 501-8808

ATLANTA Crown Office Suites 1870 The Exchange Suite 100 Atlanta, GA – 30339. Phone: +1 (770) 989-7302

United Kingdom LONDON Indium Software 71-75 Shelton Street London, WC2H 9JQ.

MALAYSIA KAULA LUMPUR Suite 8-1 & 8-2, Level 8, Menara CIMB No.1, Jalan Stesen Sentral 2 Kuala Lumpur – 50470. Phone: +60 (3) 2298 8465 Fax: +60 (3) 2298 8201

PRINCETON Carnegie Center Suite 150, 300 Carnegie Center Princeton, NJ – 08540. Phone: +1 (609) 786-2423

INDIA CHENNAI VDS House 41, 2nd Floor, Cathedral Road Chennai – 600 086. Phone: +91-44-2811 6330 Fax: +91-44-4210 4033

BENGALURU Salarpuria Adonis, 2nd Floor Binnamangala 3/1, Old Madras Road Kadiranapalya, Indira Nagar Bengaluru – 560 038. Phone: +91-80-6784 7500

SALES INQUIRIES [email protected] [email protected] [email protected] [email protected] [email protected]

GENERAL ENQUIRIES [email protected] [email protected]

© Indium Software | www.indiumsoft.com