The first advice I give when I talk about measuring performance is to test your app on a lower end device.

The question is: how low end should you go? What is the performance of the devices your users are using?

By cross-referencing:

• analytics data on some of our wide-audience apps
• Geekbench device benchmarks

we can establish a baseline!

Attributing a performance score to a device

This is what Geekbench does for us. It runs some benchmarks on phones and attributes a CPU performance score. For instance, here's the result for an iPhone 13

Geekbench 5 scores are calibrated against a baseline score of 1000 (which is the score of an Intel Core i3-8100). Higher scores are better, with double the score indicating double the performance.

Single-Core vs Multi-Core

When running, an app will spawn multiple threads. For instance, an Android React Native will likely spawn:

• the JS thread, called ++code>mqt_js++/code>
• threads starting with ++code>Fresco++/code> to display images (Fresco is the RN lib used by default to display images)
• an ++code>OkHttp++/code> thread to make API calls

Below is an example of threads using the most CPU in my app, measured with this Flipper plugin

Single-Core score correlates to the processing power of one CPU core only, basically how fast you can run one single thread. Multi-core score by opposition is how well can you run multiple processes in parallel.

For mobile apps, Single-Core is particularly important, as some threads can induce heavy calculation and can't be parallelized easily:

• the UI thread, since it's the one responsible to render your app's frames at 60fps
• the JS thread for React Native apps

For instance, the Samsung Galaxy A21s, one on the most sold phones, scores ++code>170++/code> in its single core benchmark

This is roughly 10 times less than the iPhone 13!
This means that the A21s will run JavaScript 10 times slower than the iPhone 13, which is why it is crucial to test on lower end devices.

Analyzing device performance distribution in our apps

Let's check the results for some of our apps.

1st app

User count: 963k on Android / 571k on iOS
Market: the world, but mostly Europe
Data: from March 2022, includes phones and tablets

Single Core

How to read this graph? Well for instance:

• we have ~210k Android devices scoring between 100 and 150
• we have ~40k Android devices and ~20k iOS devices scoring between 500 and 550

Some more stats: ?

++table style="border-collapse: collapse; table-layout: fixed; margin-left: auto; margin-right: auto; border: 1px solid #99acc2; height: 258.833px;" width="344" height="138">++thead>++tr style="height: 51px;">++th style="width: 135.85px; height: 51px;">++/th>++th style="width: 61.3833px; height: 51px;">Android++/th>++th style="width: 66.4167px; height: 51px;">iOS++/th>++th style="width: 79.5167px; height: 51px;">All++/th>++/tr>++/thead>++tbody>++tr style="height: 27px;">++td style="width: 135.85px; height: 27px;">Mean++/td>++td style="width: 61.3833px; height: 27px; text-align: center;">318++/td>++td style="width: 66.4167px; height: 27px; text-align: center;">1228++/td>++td style="width: 79.5167px; height: 27px; text-align: center;">655++/td>++/tr>++tr style="height: 27px;">++td style="width: 135.85px; height: 27px;">Median++/td>++td style="width: 61.3833px; height: 27px; text-align: center;">271++/td>++td style="width: 66.4167px; height: 27px; text-align: center;">1311++/td>++td style="width: 79.5167px; height: 27px; text-align: center;">507++/td>++/tr>++tr style="height: 51px;">++td style="width: 135.85px; height: 51px;">33th Percentile++/td>++td style="width: 61.3833px; height: 51px; text-align: center;">154++/td>++td style="width: 66.4167px; height: 51px; text-align: center;">1103++/td>++td style="width: 79.5167px; height: 51px; text-align: center;">288++/td>++/tr>++tr style="height: 51px;">++td style="width: 135.85px; height: 51px;">25th Percentile++/td>++td style="width: 61.3833px; height: 51px; text-align: center;">136++/td>++td style="width: 66.4167px; height: 51px; text-align: center;">914++/td>++td style="width: 79.5167px; height: 51px; text-align: center;">173++/td>++/tr>++tr style="height: 51px;">++td style="width: 135.85px; height: 51px;">10th Percentile++/td>++td style="width: 61.3833px; height: 51px; text-align: center;">98++/td>++td style="width: 66.4167px; height: 51px; text-align: center;">724++/td>++td style="width: 79.5167px; height: 51px; text-align: center;">119++/td>++/tr>++/tbody>++/table>

• 1/3 of Android users have a score below 154
• 1/4 of all users have a score below 173

This is actually lower than what I expected! Which means a lot of users with low end devices are using the app, so we should ensure their experience is good.

Also, it's safe to say there's a big disparity between Android and iOS! ?
On average, iOS devices run 5 times faster!

This is why React Native apps tend to run much better on iOS (and also the reason why I tend to focus on Android when measuring performance).

Android insightS

Taking Android alone, 74.8% of users have a score below 400.

This is because a lot of lower end devices on Android are quite popular! Here's a breakdown of the top 5 Android phones for this app:

++table style="border-collapse: collapse; table-layout: fixed; margin-left: auto; margin-right: auto; border: 1px solid #99acc2; height: 213.833px;">++thead>++tr style="height: 50.8333px;">++th style="width: 150px; height: 51px;">Name++/th>++th style="width: 77px; height: 51px;">User count++/th>++th style="width: 41px; height: 51px;">Score++/th>++th style="width: 89px; height: 51px;">Release year++/th>++/tr>++/thead>++tbody>++tr style="height: 26.8333px;">++td style="width: 150px; height: 27px;">Samsung Galaxy A51++/td>++td style="width: 77px; height: 27px; text-align: center;">16.3k++/td>++td style="width: 41px; height: 27px; text-align: center;">311++/td>++td style="width: 89px; height: 27px; text-align: center;">2020++/td>++/tr>++tr style="height: 26.8333px;">++td style="width: 150px; height: 27px;">Samsung Galaxy A12++/td>++td style="width: 77px; height: 27px; text-align: center;">10.7k++/td>++td style="width: 41px; height: 27px; text-align: center;">158++/td>++td style="width: 89px; height: 27px; text-align: center;">2020++/td>++/tr>++tr style="height: 26.8333px;">++td style="width: 150px; height: 27px;">Samsung Galaxy A21s++/td>++td style="width: 77px; height: 27px; text-align: center;">10.6k++/td>++td style="width: 41px; height: 27px; text-align: center;">170++/td>++td style="width: 89px; height: 27px; text-align: center;">2020++/td>++/tr>++tr style="height: 26.8333px;">++td style="width: 150px; height: 27px;">Samsung Galaxy A10s++/td>++td style="width: 77px; height: 27px; text-align: center;">9.3k++/td>++td style="width: 41px; height: 27px; text-align: center;">135++/td>++td style="width: 89px; height: 27px; text-align: center;">2019++/td>++/tr>++tr style="height: 26.8333px;">++td style="width: 150px; height: 27px;">Samsung Galaxy A10s++/td>++td style="width: 77px; height: 27px; text-align: center;">9.3k++/td>++td style="width: 41px; height: 27px; text-align: center;">135++/td>++td style="width: 89px; height: 27px; text-align: center;">2019++/td>++/tr>++tr style="height: 26.8333px;">++td style="width: 150px; height: 27px;">Redmi Note 8 Pro++/td>++td style="width: 77px; height: 27px; text-align: center;">8.8k++/td>++td style="width: 41px; height: 27px; text-align: center;">478++/td>++td style="width: 89px; height: 27px; text-align: center;">2019++/td>++/tr>++/tbody>++/table>

Multi-Core

We actually get similar results in multi-core:

++table style="border-collapse: collapse; table-layout: fixed; margin-left: auto; margin-right: auto; border: 1px solid #99acc2; height: 234.833px;" width="290">++thead>++tr style="height: 27px;">++th style="width: 124.817px; height: 27px;">++/th>++th style="width: 76.1px; height: 27px;">Android++/th>++th style="width: 44.1333px; height: 27px;">iOS++/th>++th style="width: 44.1167px; height: 27px;">All++/th>++/tr>++/thead>++tbody>++tr style="height: 27px;">++td style="width: 124.817px; height: 27px;">Mean++/td>++td style="text-align: center; width: 76.1px; height: 27px;">1135++/td>++td style="text-align: center; width: 44.1333px; height: 27px;">2908++/td>++td style="text-align: center; width: 44.1167px; height: 27px;">1793++/td>++/tr>++tr style="height: 27px;">++td style="width: 124.817px; height: 27px;">Median++/td>++td style="text-align: center; width: 76.1px; height: 27px;">1079++/td>++td style="text-align: center; width: 44.1333px; height: 27px;">3075++/td>++td style="text-align: center; width: 44.1167px; height: 27px;">1433++/td>++/tr>++tr style="height: 51px;">++td style="width: 124.817px; height: 51px;">33th Percentile++/td>++td style="text-align: center; width: 76.1px; height: 51px;">486++/td>++td style="text-align: center; width: 44.1333px; height: 51px;">2131++/td>++td style="text-align: center; width: 44.1167px; height: 51px;">892++/td>++/tr>++tr style="height: 51px;">++td style="width: 124.817px; height: 51px;">25th Percentile++/td>++td style="text-align: center; width: 76.1px; height: 51px;">305++/td>++td style="text-align: center; width: 44.1333px; height: 51px;">1299++/td>++td style="text-align: center; width: 44.1167px; height: 51px;">386++/td>++/tr>++tr style="height: 51px;">++td style="width: 124.817px; height: 51px;">10th Percentile++/td>++td style="text-align: center; width: 76.1px; height: 51px;">776++/td>++td style="text-align: center; width: 44.1333px; height: 51px;">2213++/td>++td style="text-align: center; width: 44.1167px; height: 51px;">1079++/td>++/tr>++/tbody>++/table>

(if you're wondering, the very powerful device on the far-right is an iPad Pro)

iOS is still the winner here, even though this time, it's only 3 times faster on average. This is because even low-end Android phones tend to have a lot of cores (6 or 8) compared to iOS phones.

2nd app

Let's compare these results with another app:
User count: 1.21M on Android
Market: France
Data: from March 2022, includes phones and tablets

The results are very similar:

++table style="border-collapse: collapse; table-layout: fixed; margin-left: auto; margin-right: auto; border: 1px solid #99acc2; height: 259px;" width="225">++thead>++tr style="height: 51.0333px;">++th style="width: 118.683px; height: 51px;">++/th>++th style="width: 52.75px; height: 51px;">App 1++/th>++th style="width: 52.7333px; height: 51px;">App 2++/th>++/tr>++/thead>++tbody>++tr style="height: 27.0167px;">++td style="width: 118.683px; height: 27px;">Mean++/td>++td style="width: 52.75px; height: 27px; text-align: center;">318++/td>++td style="width: 52.7333px; height: 27px; text-align: center;">325++/td>++/tr>++tr style="height: 27.0167px;">++td style="width: 118.683px; height: 27px;">Median++/td>++td style="width: 52.75px; height: 27px; text-align: center;">271++/td>++td style="width: 52.7333px; height: 27px; text-align: center;">271++/td>++/tr>++tr style="height: 51.0167px;">++td style="width: 118.683px; height: 51px;">33th Percentile++/td>++td style="width: 52.75px; height: 51px; text-align: center;">154++/td>++td style="width: 52.7333px; height: 51px; text-align: center;">169++/td>++/tr>++tr style="height: 51.0167px;">++td style="width: 118.683px; height: 51px;">25th Percentile++/td>++td style="width: 52.75px; height: 51px; text-align: center;">136++/td>++td style="width: 52.7333px; height: 51px; text-align: center;">143++/td>++/tr>++tr style="height: 51px;">++td style="width: 118.683px; height: 51px;">10th Percentile++/td>++td style="width: 52.75px; height: 51px; text-align: center;">98++/td>++td style="width: 52.7333px; height: 51px; text-align: center;">105++/td>++/tr>++/tbody>++/table>

The most sold Android devices differ only slightly:

++table style="border-collapse: collapse; table-layout: fixed; margin-left: auto; margin-right: auto; border: 1px solid #99acc2; height: 210.833px;">++thead>++tr style="height: 51px;">++th style="width: 150px; height: 51px;">Name++/th>++th style="width: 77px; height: 51px;">User count++/th>++th style="width: 41px; height: 51px;">Score++/th>++th style="width: 89px; height: 51px;">Release year++/th>++/tr>++/thead>++tbody>++tr style="height: 51px;">++td style="width: 150px; height: 51px;">Samsung Galaxy A20e++/td>++td style="width: 77px; height: 51px; text-align: center;">33.5k++/td>++td style="width: 41px; height: 51px; text-align: center;">231++/td>++td style="width: 89px; height: 51px; text-align: center;">2019++/td>++/tr>++tr style="height: 27px;">++td style="width: 150px; height: 27px;">Samsung Galaxy A21s++/td>++td style="width: 77px; height: 27px; text-align: center;">25.3k++/td>++td style="width: 41px; height: 27px; text-align: center;">170++/td>++td style="width: 89px; height: 27px; text-align: center;">2020++/td>++/tr>++tr style="height: 27px;">++td style="width: 150px; height: 27px;">Samsung Galaxy A12++/td>++td style="width: 77px; height: 27px; text-align: center;">22.3k++/td>++td style="width: 41px; height: 27px; text-align: center;">158++/td>++td style="width: 89px; height: 27px; text-align: center;">2020++/td>++/tr>++tr style="height: 27px;">++td style="width: 150px; height: 27px;">Samsung Galaxy A51++/td>++td style="width: 77px; height: 27px; text-align: center;">19.7k++/td>++td style="width: 41px; height: 27px; text-align: center;">311++/td>++td style="width: 89px; height: 27px; text-align: center;">2020++/td>++/tr>++tr style="height: 27px;">++td style="width: 150px; height: 27px;">Samsung Galaxy A10++/td>++td style="width: 77px; height: 27px; text-align: center;">17.6k++/td>++td style="width: 41px; height: 27px; text-align: center;">200++/td>++td style="width: 89px; height: 27px; text-align: center;">2019++/td>++/tr>++/tbody>++/table>

Conclusion

Based on this evidence, it is crucial to use a low-end Android device to measure performance. If you just test your app on an iPhone 13, you will miss most of the issues that your users can have on the widely sold A21s, which runs 10 times slower.

At BAM, we chose to go for a Samsung J3 2017, scoring 107, better than ~15% of our Android users devices.