Why You Need to Test New Apps on the Right Mobile Device Mix

By Darryl K. Taft

Thirty percent of the market can be covered with 10 to 16 devices—including the Samsung Galaxy S5, iPhone 6 and HTC One M8—and are based on market adoption, market device leaders, reference devices and device characteristics. Fifty percent market coverage can be covered with 17 to 25 devices. This level of test coverage yields an increased number of devices from manufacturers such as Sony, LG, Samsung, Apple, HTC and more.

With operating system updates requested every few weeks, it’s hard to keep up with the pace of mobile. Today, iOS 7 still accounts for some 17 percent of total users out there, and should be included at the same level of iOS 8 test plans.

Does it seem like the majority of your customers have Androids or iPhones? iPhones hold twice as much share in the U.S. than in France, Germany, Italy, Spain and the U.K. Based on 360,000 users, the iPhone device family captures 35 percent of the U.S. market, as opposed to 16 percent in France, Germany, Italy, Spain and the U.K.

Android 5.0 (Lollipop) adoption rates are low in not only the U.S., but also in France, Germany, Italy, Spain and the U.K. In fact, Android KitKat (4.4.x) and Jelly Bean (4.2.x, 4.3.x) are the most commonly used versions. Lollipop is proving to be very important, however. The latest OS from Google is running on the Google Nexus devices (considered reference devices).

Android device usage is significantly different in the U.S. than in France, Germany, Italy, Spain and the U.K. This variance is reflected in the different Mobile Test Coverage Indexes for each region.

Windows Phone and BlackBerry devices account for a total of 7 percent of mobile devices in France, Germany, Italy, Spain and the U.K. and 4 percent in the U.S. These platforms continue to see low adoption rates among end users.

The market is constantly changing, with new devices added every 90 days from market leaders. The test coverage mix should consider future devices that customers will be purchasing and later using. The index will be updated quarterly to reflect these changes.