Individual or Organizational Implementation? A Quasi-experimental Study on the Impact of Different Forms of Implementation of a Well-being App

Background:
Smartphone apps have become increasingly popular in workplaces as a tool for delivering interventions. However, it is unclear whether these apps are effective in improving health and well-being. One key question is whether these apps are successful in improving health and well-being. This paper aims to address another, equally important, question, namely whether the form of implementation matters for the functioning and outcomes. Specifically, we are interested in whether usage and outcome parameters differ between organizations that implement a well-being app as an offer to individual employees (without organizational support) and those that implement the app as an organizational intervention where the app data is fed back to teams.
Methods:
This study is quasi-experimental in nature and relies on data from a mental health and well-being app provider. The app prompts users to rate their well-being on the WHO5 every two weeks and uses these ratings to assess whether an app user is in need of additional help (total N= 23728 users). The data for this study is based on registrations from participants using the app. We have access to usage rates, assessments of well-being on the WHO5 scale, and registration of whether a person is considered “at risk” due to either low or declining well-being. When a participant is identified as "at risk", they are contacted by a clinical psychologist who assesses whether the person should be referred to further treatment, which is also registered in the data (N= 9553 cases). Data from all participants are aligned so that all start at month zero, and app usage is aggregated by three-month intervals with a total follow-up time of up to 24 months. We employ statistical tests to assess the different hypotheses in our study,
ranging from simple t-tests to repeated measures ANOVA.
Results:
Results show that, the response rates of the app differ between the two set-ups. The individual set-up has a significantly more rapid decline in app-use compared to the organizational set-up. However, there appears to be no difference with regards to the average well-being assessments made in the app. When assessed as an early warning system, significantly more employees are rated as “at risk” in the organizational support set-up, compared to the individual set-up. For these "at risk" cases, significantly more from the individual set-up are referred to a psychologist, compared to the organizational set-up.
Conclusion:
The study concludes that the form of implementation and the type of delivery of an app-based intervention does matter. Not only does the form of implementation affect the usage of the app, but it also has implications for the outcomes, namely how many "at risk" participants are identified by the early warning system. More importantly, the clinical severity of cases when they are identified by the app appears to be affected as well.