In 2017, a study found that 88% of people will abandon an app due to bugs and glitches. This startling stat demonstrates, with crystal clarity, the need for developers to make their product as clean as possible – with rigorous testing procedures to eliminate any errors.
But what are the key steps that go into the testing regime? How can you, as the developer, give yourself the best possible chance of delivering an error-free app?
Well our mobile team have offered us their years of experience in this subject, mapping out the various steps to creating a watertight regime. If you’re new to app testing and are looking for help from those who’ve been there and done it, this guide will (hopefully) come in handy.
1. Phase Your Rollout to Customers
If you release your product to the public in one decisive phase – which was the norm under the old ‘waterfall’ development methodology – it can be great, generating huge buzz around your product and cementing your reputation as a software genius. But it can also go badly.
If the app goes out with errors, they’re exposed to the full glare of public scrutiny straightaway. Your entire user base is likely to catch the same cold, and it could do major damage to your rep.
A phased release, in contrast, enables you to gradually roll out your app over a period of time. Attuned to modern agile working methodologies, which aim to get to market as soon as possible, a staged program lets you open up to a different section of your user base at each stage – and enables you to deploy analytics and monitoring tools every time.
Any issues can be easily rectified without major disruption. Then, once you’re confident that all bugs have been ironed out, you can proceed with the full release.
As well as ensuring you don’t inflict a bug-riddled update on your entire user base, the phased rollout avoids the potentially troublesome app store review. And instead of requiring a new release with each fix, a trickled feedback system enables you to hone your product piecemeal, fixing bugs as you go.
2. Utilise Analytics and Monitoring Tools
Conceptualizing an end product is one thing, but the forensic use of data will give you a detailed insight at each key stage of development – whether you’re running an iOS or Android test. Using big data analytics and monitoring tools, you can underpin the testing process with huge amounts of essential information.
From technical aspects such as crash reports, to key consumer data such as frequency of use, specific automation testing tools can help prioritise your testing. As well as allowing you to channel your energies more effectively, tools such as Firebase and New Relic provide a bird’s eye view of the app’s performance, providing information that can improve its overall look and feel.
3. Use Automation
If, like most developers, you’re constantly up against it juggling the various parts of the software build process, it can help to explore the various automation testing tools out there. We’re not suggesting you use AI for the entire testing process; it has to retain the human touch. But you can certainly use technology to relieve part of the testing burden – or handle a specific part of the process, such as user interface or user experience.
There are myriad mobile automation testing tools out there. In fact we’ve recently written blogs listing our favourite mobile-friendly test solutions for both iOS and Android. In general, through, the two we’d really recommend are Appium (which is backed by a huge open-source community) and Calabash (which allows you to write your tests in plain English, so practically anyone can take charge of the testing process).
4. Use Reliable Locators
The design of the user interface (UI) is crucial. If poorly designed, it can be unstable, slow and prone to crash. So it’s vital you get this stage right.
The UI is normally structured with a series of containers nested within one another, and in order to find an object within a UI, you’ll need a stable locator.
5. Build Easily Maintained Tests
Test maintenance can be time-consuming. That’s why it’s important to analyse what works and what doesn’t when developing a reliable testing strategy. A well-designed procedure will need fewer tweaks and can relied upon to carry out its key function, time and time again.
Each test should be broken down into a simple sequence of steps, with clear expectations mapped out. Also, you need to factor in the test environment – unreliable servers and network issues can affect test stability.
Remember: the more complex a test case, the harder it is to pinpoint problems. Also, as we suggested earlier, using iOS or Android app automation testing tools – and, more to the point, iOS or Android app development tools – can really lessen the load and break things down for you.
6. Harness Data-Driven Testing
With this form of testing, data that is external to your tests, say from an excel spreadsheet, is loaded and used to extend automated test cases. The test case is repeated automatically and the test can compare results and values to determine whether the test case passes.
Data-driven testing radically improves the accuracy and speed of results. Even the most scrupulous expert is prone to making errors when manually testing, but this method makes it possible to cover large amounts of repetitive testing with pinpoint accuracy, freeing up time and resources to improving the user experience.
Well-designed data-driven tests simplify the whole process, reducing test case maintenance and enabling easy modification.
7. Combine Functional Testing and Stress Testing
While functional testing assesses the design integrity and usability of the app, it does so under ideal conditions. As any developer will tell you, it’s rare that you get these in real life. So it’s vital to find out how your app will stand up to the stresses and strains of the real world, and your testing regime needs to be tailored accordingly.
A good stress test will push your app to its breaking point. It should also expose issues that wouldn’t crop up under normal conditions, so you can then fix them – before they affect your end users.
Timing is important here: it’s best to leave the stress test right up until you’re ready to go live with your product, or update. If, like many developers, you’ve adopted an agile methodology, leave your stress test until the hardening sprint (the additional, final sprint you run when all the other tests are complete).
When the stress test is ready, you should carry it out on real devices, not simulators. What’s more, if you’re working on Android, you should aim to test on all the major devices in the market if possible.
Make sure you deal with all the possible complications. For example, you can increase the latency of the network and the error rate (the number of errors your device is subjected to during the test), and see whether your app stands up to these changes.
When carrying out the test, it’s vital to factor in all the common circumstances that can cause problems Check how your app performs offline, or in the middle of a http request – paying attention to whether it caches and restores correctly. Examine closely what happens when the app is killed while it’s in the background. And, of course, make sure you subject it to the old ‘key-bashing test’, as many of your users are likely to be – how do we put this – heavy-fingered.
In addition, try and throw in a few random occurrences which could throw a spanner in the works. To help you with this, we recommend you take a look at Monkey. It’s a tool that comes free with the Android SDK and generates loads of pseudo-random events to test your app.
8. Plan UI Testing
UI testing is a comprehensive dissection of a system, from start to finish, ensuring it functions and flows perfectly under real-world scenarios. UI testing is the last link in the testing chain and, therefore, it’s crucial.
Because of its importance, UI testing is a complex, delicate operation that is difficult to maintain. It can be time-consuming to undertake manually (automated UI testing is very expensive) and involves many steps with minute details.
Testing in the correct order, managing the test environment and creating an organised UI plan – using test cases designed with an end-user perspective – are all crucial in order to create a great mobile app.
9. Keep Your Focus on the Customer Experience
Throughout the meticulous testing process it can be easy to forget that, in the end, it all comes down to the customer. While each of the previous tips is vital in and of itself, it’s vital to keep the end user’s perspective in mind at all time.
Whether it’s making use of a phased rollout to test the app and avoid customer satisfaction, creating a simulated test environment to determine a real-world setting or simply using monitoring tools to find out what your user wants, it’s important to really go the extra mile on this – because, no matter how tight and error-free your app is, it’s the market that’ll determine its success or failure.
10. Check Out Bugfender
Ok, we decided to add this final one because, well… it’s our list, right? We can do what we want!
Before you go, we wanted to mention our key product, Bugfender, which combines crash reporting and remote logging. So as well as telling you when your app has broken down (which isn’t very helpful in terms of proactive planning), Bugfender monitors your app’s performance and allows you to see problems coming down the pipe.
We appreciate there are loads of great testing products out there (we’ve tried to mention as many of them as we can), but Bugfender can really aid your testing regimen, either complementing some of the aforementioned tools or offering an alternative.
