In the world of mobile applications and digital products, quality assurance (QA) is critical. Good quality assurance practices are necessary to make sure an application delivers an optimized customer experience before it’s released to the masses. For agencies, quality assurance programs can be a real competitive differentiator that helps win new business and customer loyalty. For brands, having high QA standards is often what stands between them and the bad reviews and social media backlash that a buggy product can generate.
However, a close look at the QA practices at many companies quickly reveals that not all QA is created equal. The wild proliferation of mobile devices means QA today is more challenging and more critical than ever and many companies are simply not up for the task. With thousands of different devices, hundreds of browser versions, and dozens of operating systems in the market, testing for quality can be a daunting undertaking. In fact, a report by Flurry revealed that to reach 80 percent of active mobile users last year, developers had to support 156 devices and to reach 90 percent, a whopping 331. Are the digital product agencies you are selecting really doing the quality assurance testing on these growing numbers of devices to protect the reputation of your brand?
While it is often difficult to fully assess if your agency is following the proper QA guidelines, there are several criteria you should incorporate into your vendor evaluation process. Here are five techniques we demand from our QA team at L4 Digital that you should demand from any company that you entrust your brand’s reputation to:
- Broad device testing: To validate their stability and functionality, apps need to prove themselves across a spectrum of devices from iPhones to Android tablets and everything in between. The first thing you should require from your agency is an examination of their device testing process across a defined device group. Let your agency know you want an accounting of the devices they are testing against and make sure the ones most important to your customer base are on the list. At a minimum, L4 tests more than 50 different combinations of the most popular device, OS and browser types and the list seems to grow with every new release.
- Customized test plans: The more that testing can reflect actual customer usage, the more effective it will be. As a second element of the QA process, it is important to have documentation and use cases which can be applied to create custom feature testing plans. Many agencies will take a shortcut and just test against basic application functionality (e.g. Did the app turn on?, Does it close properly?, etc.) You want to make sure your agency is testing against your application’s unique features and functionality. Those are the elements that matter most to your customers and target audiences and will be critical to maintaining your reputation once the application is in market.
- Automated testing: Automated testing is a third area that many of the more progressive agencies now implement in earnest. With so many devices in market it isn’t feasible for any agency to manually test all the variants necessary to release a quality produce in a reasonable time frame. Automated testing helps tackle the scalability of testing issue and is often broken into two tiers:
Device testing automation that installs applications across different device types and automatically clicks on the screen to make sure basic installation and load requirements are met. At a minimum, most agencies you choose should perform this function.
Feature testing automation that offers the depth of testing to ensure the appropriate custom verification of product specific user stories and features are in place. This practice assures that performance is as intended on an array of screen sizes, hardware types and OS versions. Automated feature testing checks the most important features to make sure they work correctly regardless of screen size, device types, etc.
Feature testing automation should tie back to the requirements in the scope of work to make sure all the products you deliver work as planned. At L4 we also review acceptance criteria documentation from all app stores and include those criteria in our automated testing as well. Time is often of the essence when it comes to application launches and high quality automated feature testing against known acceptance criteria can make sure your app gets into the stores for distribution the first time it is submitted.
- Manual testing: There is simply no substitute for human perception and judgment. Automation should always be supplemented with human manual testing, especially on the most critical devices for your market. It’s important to run manual tests on individual phones to make sure that core app features match requirements, and that the app looks and feels the way it’s supposed to. Make sure the agency you choose manually tests your most important product features, especially the ones that require complex interactions that can’t be tested through automation.
- End-to-end regression testing and reporting: Many companies only test features individually as they’re being developed. Once the application is complete, time constraints often squeeze out testing time. At L4, we always perform a complete end-to-end test after all the features have been assembled together. This is the only way to evaluate if all the features are working correctly together to deliver the desired brand experience.
We also provide a report of all QA test results we record so our customers know exactly what was evaluated. If your agency isn’t being open about their QA testing results, I would assume that most tests are being performed during the development process and not after the product is feature complete. That shouldn’t be acceptable for your company or your customers.
Rigorous quality assurance practices are critical to the success of a product. Once an app is in the marketplace, it gets much harder to make changes. You can release post-launch updates but there is no guarantee that people will install them. Getting QA right at the outset is critical to the success of your app and directly impacts your brand’s reputation. If your vendor doesn’t have a solid QA plan that incorporates the 5 QA techniques listed above it is time to look elsewhere. Your brand is too important and your customers deserve better.[mc4wp_form]
Image courtesy of Helloquence for Unsplash.