A systematic review of trials of web-based and computerised tools to help manage diabetes has found mixed evidence of their effectiveness.
In diabetes care, do computerised tools for patients, carers or clinicians improve psychological, behavioural or clinical outcomes?
As well as reviewing the published evaluations, each tool was assessed by experts for its clinical effectiveness, usability and sustainability in practice.
The reviewers found studies by searching bibliographic databases and online using Google. They found 92 different tools, evaluated in 57 different studies.
First, they evaluated the clinical usefulness, usability and sustainability of each tool they identified. Next, they looked at what each of the studies said about how effective they were at improving users’ knowledge, behaviours or health.
Unfortunately, the studies were too different from one another to be able to draw an overall conclusion.
Although people who used the tools more seemed to get more benefit from them, this may be because the healthier people were more able to use the tool, rather than the other way around.
The tools that were most effective were the ones that used a lot of interactivity and feedback.
The reviewers state:
Methodological quality and ratings for clinical usefulness and sustainability were variable, and there was a high prevalence of usability errors. Tools showed moderate but inconsistent effects on a variety of psychological and clinical outcomes including HbA1c and weight.
Developers of any such resource must take heed of these findings. Many tools are ineffective, and there is a pressing need for better quality, longer term evaluations of these tools that focus on objective outcomes.
- The Google search was done in 2009 and the literature search was done in June 2011. There have been important studies since then (see below).
- The review had a very broad focus on any sort of online tool. It may be more appropriate to look for good quality individual studies that evaluate the type of tool you are interested in, or the target population that is relevant to you.
- Many bibliographic databases are difficult to search effectively for this kind of tool. The authors should have included computer science, education and social science databases. For these reasons, you should consider it likely that there may be other research in this area that did not turn up in the authors’ literature search.
- Half of the studies did not perform a proper comparison of the tool against an alternative.
- Questions may be asked about the details of usability assessment. The reviewers used indirect expert evaluation of tool usability rather than direct empirical evaluation.
The full text of this paper is available from the journal website:
Yu CH, Bahniwal R, Laupacis A, Leung E, Orr MS, Straus SE. Systematic review and evaluation of web-accessible tools for management of diabetes and related cardiovascular risk factors by patients and healthcare providers. J Am Med Inform Assoc. 2012 Jan 3.