Mobile application testing

Mobile application testing is a process by which application software developed for handheld mobile devices is tested for its functionality, usability and consistency.^[1] Mobile application testing can be an automated or manual type of testing.^[2] Mobile applications either come pre-installed or can be installed from mobile software distribution platforms. Mobile devices have witnessed a phenomenal growth in the past few years. A study conducted by the Yankee Group ^[3] predicts the generation of $4.2 billion in revenue by 2013 through 7 billion U.S. smartphone app downloads.

Additionally, wearable application testing is an interesting market. Bluetooth, GPS, Sensors, Wi-Fi are some of the core technologies at play in wearables. A lot of importance is needed here for field testing, user focus, and looking at areas where hardware and software need to be tested in unison.

Key Challenges for Mobile Application Testing

1. Must be Downloadable: Should be Downloadable from Particular Store

2. Diversity in Mobile Platforms/OSes: There are different mobile operating systems in the market. The major ones are Android, iOS, Symbian, Windows Phone, and BlackBerry (RIM). Each operating system has its own limitations. Testing a single application across multiple devices running on the same platform and every platform poses a unique challenge for testers.

3. Device Availability: Access to the right set of devices when there is an ever-growing list of devices and operating system versions is a constant mobile application testing challenge. Access to devices can become even more challenging if testers are spread across different locations.

4. Mobile network operators: There are over 400 mobile network operators in the world;^[4] out of which some are CDMA, some GSM, whereas others use less common network standards like FOMA, and TD-SCDMA. Each network operator uses a different kind network infrastructure and this limits the flow of information.

5. Scripting: The variety of devices makes executing a test script (scripting) a key challenge. As devices differ in keystrokes, input methods, menu structure and display properties single script does not function on every device.

6. Choosing how to test: There are two main ways of testing mobile applications: testing on real devices or testing on emulators. Unfortunately, neither method is flawless.^[5] Emulators often miss issues that can only be caught by testing on real devices, but because of the multitude of different devices in the market, real devices can be expensive to purchase and time-consuming to use for testing.^[6][7]

7. Compatibility: It is necessary to test the Compatibility: Suppose an Application can work on the high resolution and it doesn't work on the fewer lower resolution.

8. Should be able to Pick up the Phone: It is necessary to check: While executing the app Application should be able to pick up call.

9. Variety of Mobile Devices: Mobile devices differ in screen input methods (QWERTY, touch, normal) with different hardware capabilities.

Types of Mobile Application Testing

1. Functional Testing:testing Functional testing ensures that the application is working as per the requirements. Most of the test conducted for this is driven by the user interface and call flow

2. Laboratory Testing: Laboratory testing, usually carried out by network carriers, is done by simulating the complete wireless network. This test is performed to find out any glitches when a mobile application uses voice and/or data connection to perform some functions.

3. Performance Testing: This testing process is undertaken to check the performance and behavior of the application under certain conditions such as low battery, bad network coverage, low available memory, simultaneous access to application’s server by several users and other conditions. Performance of an application can be affected from two sides:application’s server side and client’s side. Performance testing is carried out to check both.

4. Memory Leakage Testing: Memory leakage happens when a computer program or application is unable to manage the memory it is allocated resulting in poor performance of the application and the overall slowdown of the system. As mobile devices have significant constraints of available memory, memory leakage testing is crucial for the proper functioning of an application

5. Interrupt Testing: An application while functioning may face several interruptions like incoming calls or network coverage outage and recovery. The different types of interruptions are:

• Incoming and Outgoing SMS and MMS
• Incoming and Outgoing calls
• Incoming Notifications
• Battery Removal
• Cable Insertion and Removal for data transfer
• Network outage and recovery
• Media Player on/off
• Device Power cycle

An application should be able to handle these interruptions by going into a suspended state and resuming afterwards.

6. Usability testing: Usability testing is carried out to verify if the application is achieving its goals and getting a favorable response from users. This is important as the usability of an application is its key to commercial success (it is nothing but user friendliness).^[8] Another important part of usability testing is to make sure that the user experience is uniform across all devices.^[9] This section of testing hopes to address the key challenges of the variety of mobile devices and the diversity in mobile platforms/OS, which is also called device fragmentation. One key portion of this type of usability testing is to be sure that there are no major errors in the functionality, placement, or sizing of the user interface on different devices.,^[10]

7. Installation testing: Certain mobile applications come pre-installed on the device whereas others have to be installed from the store. Installation testing verifies that the installation process goes smoothly without the user having to face any difficulty. This testing process covers installation, updating and uninstalling of an application

8. Certification Testing: To get a certificate of compliance, each mobile device needs to be tested against the guidelines set by different mobile platforms.

The Certified Mobile Application Tester (CMAT) certification exam is offered by the Global Association for Quality Management (GAQM) via Pearson Vue Testing Center worldwide to benefit the Mobile Application Testing Community.

9. Security Testing: To check for vulnerabilities to hacking, authentication and authorization policies, data security, session management and other security standards.

10. Location Testing: Connectivity changes with network and location, but you can't mimic those fluctuating conditions in a lab. Only in Country non automated testers can perform comprehensive usability and functionality testing.

11. Outdated Software Testing:ost: Not everyone regularly updates their operating system. Some Android users might not even have access to the newest version. Professional Testers can test outdated software.

12. Load Testing: When many users all attempt to download, load, and use your app or game simultaneously, slow load times or crashes can occur causing many customers to abandon your app, game, or website. In-country human testing done manually is the most effective way to test load.^[11]

13. Black box Testing: This type of testing doesn't include the internally coding logic of the application. Tester tests the application with functionality without peering with internally structure of the application. This method of test can be applied virtually to every level of software testing: unit, integration, system and acceptance.

14. CrowdSourced Testing: In recent years, crowdsourced testing has become popular as companies can test mobile applications faster and cheaper using a global community of testers. Due to growing diversity of devices and operating systems as well as localization needs, it is difficult to comprehensively test mobile applications with small in-house testing teams. A global community of testers provides easy access to different devices and platforms. A globally distributed team can also test it in multiple locations and under different network conditions. Finally, localization issues can be tested by hiring testers in required geographies. Since real users using real devices test the application, it is more likely to find issues faced by users under real world conditions. ^[12]

References