A Systematic Review of the Methodologies Used to Evaluate Telemedicine Service Initiatives in Hospital Facilities

Background: The adoption of telemedicine into mainstream health services has been slower than expected. Many telemedicine projects tend not to progress beyond the trial phase; there are a large number of pilot or project publications and fewer 'service' publications. This issue has been noted since 1999 and continues to be acknowledged in the literature. While overall telemedicine uptake has been slow, some services have been successful. The reporting and evaluation of these successful services may help to improve future uptake and sustainability. The aim of this literature review was to identify peer-reviewed publications of deployed telemedicine services in hospital facilities; and to report, and appraise, the methodology used to evaluate these services.

Methods: Computerised literature searches of bibliographic databases were performed using the MeSH terms for "Telemedicine" and "Hospital Services" or "Hospital", for papers published up to May 2016.

Results: A total of 164 papers were identified, representing 137 telemedicine services. The majority of reported telemedicine services were based in the United States of America (n=61, 44.5%). Almost two thirds of the services (n=86, 62.7%) were delivered by real time telemedicine. Of the reviewed studies, almost half (n=81, 49.3%) assessed their services from three different evaluation perspectives: clinical outcomes, economics and satisfaction. While the remaining half (n=83, 50.6%) described their service and its activities without reporting any evaluation measures. Only 30 (18.2%) studies indicated a two-step implementation and evaluation process. There was limited information in all reported studies regarding description of a structured planning strategy.

Conclusion: Our systematic review identified only 137 telemedicine services. This suggests either telemedicine service implementation is still not a part of mainstream clinical services, or it is not being reported in the peer-reviewed literature. The depth and the quality of information were variable across studies, reducing the generalisability. The reporting of service implementation and planning strategies should be encouraged. Given the fast paced technology driven environment of telemedicine, this may enable others to learn and understand how to implement sustainable services. The key component of planning was underreported in these studies. Studies applying and reporting more rigorous methodology would contribute greatly to the evidence for telemedicine.