Why health app research matters
9 min read
The development of mobile apps has revolutionized the way we live and interact with the world. Apps are no longer just simple tools or games, but complex systems that can have a profound impact on our lives. From managing our finances to tracking our fitness, there are now apps for almost everything. Given their widespread use and potential impact on health, it is essential that apps are rigorously researched.
Health app research is important for a number of reasons. It helps ascertain that they are both safe for the health of users and of good clinical value, providing tangible health benefits. Secondly, it provides evidence which can inform the decisions of insurance companies and even some governments to reimburse certain apps. Finally, clinical research can help define patient groups that derive the most benefits from the app and under which circumstances, allowing developers to optimize their tools.
Research is important, but not any research
There are a number of criteria that must be met in order for research to be considered scientifically valuable. Firstly, the study must have been conducted using an appropriate methodology. The results of the study must have been published in a peer-reviewed journal. Also, the study must have been conducted on a large enough sample size, with a sufficient effect size and low variability for the difference to be considered significant. Finally, the findings of the study should be generalizable to a wider population.
Appropriate methodology is essential in order for research to be considered scientifically good. The methods used should be appropriate for the question being asked and should follow internationally recognized guidelines. Ideally, a protocol should be published prior to initiating the study in order to prove that hypotheses and methods were prespecified and not altered by examining the data. The results of the study should also be published in a peer-reviewed journal to ensure they have undergone rigorous scrutiny by experts in the field.
The sample size of a study is also important when determining its scientific validity. A small sample size can lead to false positives or false negatives and may not be representative of the wider population. Studies with large sample sizes are more likely to produce reliable results that can be generalized to a larger population. Optimal sample size calculation that minimizes uncertainty should be performed when designing the study.
The type of study is also an important consideration. For example, randomized double-blind studies are highly valued in research because they provide more reliable and unbiased results. Double-blind studies are often considered the most rigorous method for evaluating the effectiveness of a treatment or intervention. In the field of app research, because in this type of study neither the participants nor the investigators know who is receiving the experimental or control treatment, it will mean that an app evaluated in this manner will need to be compared to another app; this will then eliminate preconceived expectations or biases that may affect the results.
When choosing or recommending an app, consider more those with studies
There are a number of advantages to choosing an app that has been subjected to scientific research. First and foremost, you can be more confident that the app is safe and effective, and will help you, or your patient, reach the expected health goals. If you are a health professional, it is standard practice for you to recommend treatments that have been studied and proven by science. You will be interested to know the expected benefits, but also the possible risks and side effects of a treatment for your patients.
Also, the evidence generated by research makes it more likely that insurance companies and governments will reimburse the cost of the app. Even if the app is not reimbursed, your money (or your patient's money) is probably better spent than even a lesser amount on an app that has not been studied.
Studies and access
Whether a treatment is covered or not, as well as the patient's ability and willingness to pay for a treatment, is an important consideration. Thus, it is interesting to know which apps have proven their effectiveness through studies and thus have a better chance of being reimbursed by certain insurers.
In some countries, public insurance plans are starting to reimburse certain apps upon prescription, but unfortunately this is not the case in Canada. On the other hand, private plans may reimburse some apps if they have been shown to provide benefits to patients. Private companies may also cover the cost of certain apps, or unlock premium features in apps, for their employees.
TherAppX Review process
At TherAppX, our process is to search for the presence of studies in 4 categories for each app:
Acceptability and/or feasibility studies
Peer-reviewed clinical value studies
Peer-reviewed economic value studies
Other studies, which may include, for example, studies on a connected device, but not evaluating the associated app itself.
We'll discuss the importance of the first three types.
Acceptability and/or feasibility studies
These are two types of research that are often conducted on apps.
Acceptability studies are important because they help to determine whether users will actually use the app, and collect their input. If an app is not acceptable to users, it is unlikely that they will stick with using it, regardless of how effective it may be.
Feasibility studies, on the other hand, help to determine whether an app and its features can actually be used as intended. If an app is not feasible to use, then it will not be able to deliver its intended results.
Both types of research play a crucial role in determining whether or not an app will ultimately be successful. Without them, there would be no way of knowing whether or not users would actually use the app or if the app would even work as intended.
Peer-reviewed clinical studies
These studies have been conducted using an appropriate methodology, with the results published in a peer-reviewed journal. These studies are important because they provide evidence of the clinical value of apps, that is, the extent to which an app has a real impact on a patient's life. They often compare two groups, and assess whether there is a significant improvement of one of the evaluated outcomes; this improvement is often given as a percentage, either relative or absolute.
Often, these studies will determine whether the app can be sold and prescribed as a regulated medical device in a particular country. Furthermore, they can influence the decisions of insurance companies and even some governments to reimburse certain apps.
Peer-reviewed economic value studies
Economic value studies are important because they help to assess the cost-effectiveness of an app. They are usually peer-reviewed if they have been published in a scientific journal. As they help decision-makers make informed choices about how to allocate resources and invest in areas that will generate the greatest economic and financial returns, such studies are often conducted for health insurance companies and governments to determine whether or not an app is worth reimbursing (once it was shown to be clinically effective).
If you want to understand studies even more
There are some concepts that can help you make up your mind when analyzing the methodology and results of a study:
Use of a control group
In research, the control allows a comparison for evaluating the relative effects of an intervention or treatment. Specifically, it can be a control group that is given as an inactive intervention, or a comparable intervention to the one that is considered standard of care.
The purpose of using a control is to provide a valid comparison to assess the actual effects of the intervention. By comparing the results between the group that received the intervention and the control group, it is possible to determine whether the observed effects are attributable to the intervention or are simply due to other factors.
A priori hypothesis
In research, an a priori specified hypothesis is one that is formulated before data collection and is explicitly tested in the study.
The advantage of determining an a priori hypothesis is that it helps to clarify the objectives of the study and establish criteria that will be used to assess the validity of the findings. It can also help to minimize bias in data collection and analysis by focusing attention on data relevant to the hypothesis being tested.
Allocation and blinding methods
These methods are important strategies for promoting the validity of a study's results. Allocation refers to how participants are assigned to different treatment groups, while blinding refers to how participants and researchers are informed of the treatment that subjects are receiving.
Good allocation methods include, for example, the use of randomization, to minimize selection bias and ensure that each participant has an equal chance of being in one treatment group or the other such that the composition of each group is balanced.
With respect to blinding, there are several types of blinding, such as participant, investigator, evaluator, and data blinding. Good blinding methods involve hiding from participants and the research team the treatment that is being administered. This can be accomplished using placebos, sham treatments, or by hiding the actual treatment. Data blinding also involves hiding the treatment that was administered even while analyzing the results.
Good allocation concealment and blinding prevent results from being influenced by preconceptions and biases originating from both study participants and investigators.
Statistical methods to determine a significant effect
Statistical methods in clinical trial design and conduct can get very complex. Overall, the goal of these methods is to ensure that a certain level of confidence is associated with each result.
A difference in results is considered "statistically significant" if it is highly unlikely to appear by chance alone, according to a prespecified probability margin.
The clinical significance, that is, the meaningfulness of a difference when translated into clinical practice, varies for each outcome measurement and can be influenced by the clinical setting. The implication of this other, more realistic, margin is that a small, even though statistically significant difference, may not have a tangible impact on the patient's life. Hasty statistical conclusions must therefore be interpreted with caution.
Here are some examples of apps for which a good number of robust studies have been identified.
To quit smoking
Smoke Free is an app to help quit smoking. The user has access to professional advice to help them quit, as well as a tracker to monitor their progress. Smoke Free also provides badges and health improvement bars to keep the user motivated.
For this app, there are both positive acceptability/feasibility, clinical and economic studies, all peer-reviewed.
To sleep better
Sleepio is an app designed to help users with sleep-related issues get better sleep. It uses cognitive behavioral techniques to help users change their thinking and behavior around sleep, which encourages better sleep habits. The app is backed by clinical research and proven to help users fall asleep faster, spend less time awake at night, and have better functioning during the day.
For this app, there are both positive acceptability/feasibility, clinical and economic studies, all peer-reviewed.
To lose weight
Noom is a weight loss app that helps users lose weight and get healthy. The app features a psychology-based weight loss course, state-of-the-art tools, and group support. Noom users lose an average of 15.5 pounds over 16 weeks.
For this app, there are positive peer-reviewed acceptability/feasibility and numerous clinical studies.
To relax
Calm is a popular app for sleep, meditation, and relaxation. It helps manage stress, balance moods, and improve sleep quality. The app features guided meditation, sleep stories, soundscapes, and breathwork exercises. It also has a library of relaxing music and sounds.
For this app, there are both positive acceptability/feasibility, clinical and economic studies, all peer-reviewed.
To improve global well-being
SilverCloud is an app that helps manage mental and behavioral health issues. It provides a wide range of supportive and interactive programs, tools and tactics. These programs address wellbeing, life balance, time management, communication skills, goal setting, communication and relationship management, anger management, stress management, relaxation and sleep management, among many others.
For this app, there are both positive acceptability/feasibility, clinical and economic studies, all peer-reviewed.
What to take away
App research is essential in order to ensure that apps are safe and effective. Good research methodology, large sample sizes, and peer-review are important factors to consider when determining the scientific validity of a study.
Peer-reviewed acceptability and/or feasibility, clinical value and economic studies, when available, all add valuable information when you're looking for the right app for your needs or goals.
Whether you are a healthcare professional or a user who is interested in self-care, when it comes to choosing an app, it may sure be advantageous to select one that has been subjected to scientific research!
Your health care professional can help you choose
Feeling confident is important. Get all the essential information about health apps by talking to your healthcare professional.
AppGuide provides reliable information about mobile health apps that allows patients and healthcare professionals to make informed, shared decisions about using a health app to track health status or act on your priority health goals.