Android 11 beta watch, part 1

Android 11 beta watch (beta 1 & beta 2)

Summary

In the new Android 11 most applications work just as well as in Android 10, but frequent crashes when using the drawer and volume controls indicate these are still very much beta releases. This type of constant Drawer issues are very annoying for the users.

How we tested

We designed 100 different test cases that covered common use cases such as making and receiving calls, taking photos and browsing the internet. These test cases were then automated and run for roughly 100 hours continuously for both beta releases and on both Pixel 3 and Pixel 4. The objective was to find out what kind of stability problems real life end users (you and me) can expect in long term usage. No modifications were done to the hardware or software of the Phones apart from setting the developer options on to enable USB connection.

Key findings

Drawer crashes

By far the biggest problems we saw were frequent crashes when pulling up the notification drawer. This would occasionally also happen when the volume dialog would come up by changing volume using the volume keys. The UI first freezes and after 5 seconds, the Android OS stops either the launcher (com.android.launcher) or the main UI process (com.android.systemui). Even though a different app is reported these bugs likely share the same root cause.

Unstable camera

There were several different types of crashes with camera features. Most severe of these was a bug, that caused the camera to just exit when the user attempted to start recording a video. Fortunately, these seem to have been fixed for beta 2 as we only detected 1 crash compared to 20+ in beta 1.

Tune in next week when we follow up with the results from Android 11 beta 3.

You can follow us on LinkedIn or sign up for our newsletter at the bottom of this page.