Mixed-Method Usability Evaluation of Diabetes Self-Care Applications (MySugr, Glucose Buddy and Glooko)

Abstract

Objective: This study aimed to test the usability of three top-rated diabetes management applications for Android and AppStore - MySugr, Glucose Buddy, and Glooko. Nine participants with T2D were involved in the testing. The main goal is to apply usability
evaluation methods (UEM) to rate these applications and to provide guidelines for selecting appropriate UEM for similar studies. Additionally, to answer the sub-question, the functional design requirements of mobile diabetes applications considered essential by participants during the interview.
Background: The digital market is overloaded with mobile applications for health and lifestyle improvements. However, the usability of the majority of these applications remains low. Users often struggle to find an application that suits their specific needs. It leads to the lower number of active users and risks to revenue reduction of vendors. Therefore, usability evaluation (UE) is crucial for application development, enabling vendors to monitor user satisfaction levels.
Methods: The data collection employs a mixed-method approach, combining qualitative and quantitative methods. Considering available resources and the testing stage of the development cycle, it is chosen to use a testing and inquiry types of UEM, which includes qualitative semi-structured interviews, the mixed Think-Aloud Protocol (TAP), and a quantitative System Usability Scale (SUS) questionnaire. Additionally, the TAP results are measured using the success rate metrics. The higher the success rate and SUS score results, the higher the application's usability is ranked. For the appropriate UEM selection, a literature search is performed using a visual road map to summarize the results.
Results: Glucose Buddy has the highest usability rating, with a success rate of 89 and a SUS score of 85, compared to MySugr, with a success rate of 83 and a SUS score of 70, and Glooko, with a success rate of 72 and a SUS score of 56.6. Based on the measurements conducted, it has been determined that both Glucose Buddy and MySugr have noticeably higher usability than Glooko. Initially, MySugr was more prevalent among users than Glucose Buddy, with higher Monthly Active Users (MAU) and monthly downloads. However, according to the results received, popularity does not necessarily guarantee a high usability rating. Research shows that all three applications had usability issues. This study also involves a literature search on UE selection methods. As a result, a methods selection roadmap was created. The selected methods must be suitable for the testing stage of the development cycle, involve actual users, have a mixed qualitative and quantitative approach, and be simple to conduct. In addition, during the semi-structured interview, participants outlined their design requirements for mobile diabetes applications. They emphasized the importance of multifunctionality, simple and intuitive design, customizable features, good interoperability, affordability, and time efficiency by automating routine manual processes. Furthermore, participants highlighted the positive impact of community support on their diabetes management standards.
Conclusion: The study found that Glucose Buddy had the best usability compared to the second-ranking app, MySugr. Glooko's results fell below the benchmarks. However, based on interview observations, there is still room for positive adjustments for all three apps. Additionally, the study unveiled new findings about design requirements for diabetes mobile applications and the lack of UAM selection guidelines, necessitating future research.