«Evaluating Smart Forms and Quality Dashboards in an EHR Inclusive Dates: 09/30/04 - 09/29/09 Principal Investigator: Blackford Middleton, MD, MPH, ...»
Grant Final Report
Grant ID: HS015169
Evaluating Smart Forms and Quality Dashboards
in an EHR
Inclusive Dates: 09/30/04 - 09/29/09
Blackford Middleton, MD, MPH, MSc
Jeffrey A. Linder, MD, MPH John Orav, PhD
Jeffrey L. Schnipper, MD, MPH Yelena Kleyner
Jonathan Einbinder, MD Julie Fiskio
Lynn Volk, MHS Ruslana Tsurikova, MA, MSc Tonya Hongsermeier, PHD Tony Yu Qi Li, MSc
Brigham and Women’s Hospital
The Agency for Healthcare Research and Quality (AHRQ) U.S. Department of Health and Human Services 540 Gaither Road Rockville, MD 20850 www.ahrq.gov Abstract Purpose: 1. To design and implement an integrated documentation-based clinical decision support and physician feedback system, provided in an electronic health record (EHR), to improve the management of patients with acute and chronic medical conditions. This study focused on three conditions - acute respiratory tract infections (ARI), coronary artery disease (CAD) and diabetes mellitus (DM). 2. To assess the perceived value of EHR quality dashboards by clinicians and their marginal impact over smart forms on compliance with best practices in ARI and CAD.
Scope: Smart Forms integrate decision support into clinical documentation templates, thus facilitating clinical decision support, ordering, and patient education. The current version of the Smart Form is designed around two clinical areas: acute respiratory tract infections (ARI) and coronary artery disease (CAD)/diabetes mellitus (DM). A second application we developed was the Quality Dashboards (QD). Quality Dashboards track statistical data about patient care, in order to evaluate how closely clinicians follow guidelines on best practices. They are also meant to display patient data in order to track and benchmark physician and practice performance against other physicians and practices, within a specified community. Smart Forms and Quality Dashboards were introduced to as many as 27 Partners affiliated primary care clinics and were used by over 400 clinicians in the course of a randomized control study.
Methods: Smart Forms and Quality Dashboards were designed and developed by the research team in conjunction with Partners software developers. Four Randomized Control Trials (RCT) randomized by practice were conducted that compared usual care, use of Smart Forms alone, and use of Smart Forms plus Quality Dashboards. A smaller scale pilot preceded three out of four trials to access feasibility. The pilot users were asked to fill out a survey following the pilot and major barriers were addressed prior to each large-scale RCT. The difference between the intervention practices and control practices served as the outcome measures. In addition, we identified and addressed clinician and system barriers to the effective use of Smart Forms.
Statistical software packages SUDAAN and SAS were used to analyze the data.
Results: ARI Smart Form study revealed a small but significant difference in antibiotic prescribing rates. In the intent-to-intervene analysis, clinicians prescribed antibiotics to 43% of patients with ARI diagnoses in control clinics and to 39% of patients with ARI diagnoses in intervention clinics. Usage data from the ARI QD pilot indicates that pilot users accessed the application to run reports to see how he or she performed on antibiotic prescription rates compared to his or her practice peers and against national benchmarks. Pilot data also indicated that clinicians found the ARI QD with information on diagnoses and levels of service billing data comparisons integral to understanding practice patterns for ARI. CAD/DM Smart Form RCT ran for 310 days and involved 239 physicians and during over 26,000 visits. All measures data has been collected and is presently in final stages of analysis. CAD/DM Smart Form pilot suggested a trend towards improved participants’ satisfaction with their management of smoking, ACE I/ARB use, and especially diet and exercise, but these differences were not statistically significant. In the post-pilot usability survey, the majority of participants agreed that the CAD/DM Smart Form helped them to improve compliance with clinical guidelines and improve the quality of patient care. CAD/DM QD RCT was conducted with 15 primary care clinics (8 in the intervention group and 7 in control). All measures data for the study was retrieved by May, 2009 and is being analyzed. Results from the user survey indicate that exposure to new CDS even without actual use may marginally increase adherence to the clinical guidelines.
Key Words: coronary artery disease, Diabetes Mellitus, acute respiratory tract infections, clinical decision support, smart form, quality dashboard The authors of this report are responsible for its content. Statements in the report should not be construed as endorsement by the Agency for Healthcare Research and Quality or the U.S.
Department of Health and Human Services of a particular drug, device, test, treatment, or other clinical service.
This research project was funded to evaluate documentation-based clinical decision support and quality dashboards through the following three project aims.
Aim 1. To design and implement an integrated documentation-based clinical decision support and physician feedback system, provided in an electronic health record (EHR), to improve the management of patients with acute and chronic medical conditions.
Hypothesis 1. A documentation-based clinical decision support tool (CDSS) “smart form”, and a “quality dashboard” physician feedback system, can be designed and implemented to facilitate documentation and physician order-entry, provide individualized, evidence-based recommendations for the management of patients with coronary artery disease (CAD) and acute respiratory tract infections (ARIs), and are usable by primary care physicians.
Aim 2. To determine the effectiveness of documentation-based CDSS and physician feedback on documentation and the clinical management of patients with coronary artery disease and acute respiratory tract infections.
Hypothesis 2A. A documentation-based CDSS “smart form” will increase the documentation of important clinical data in patients with CAD and ARI when compared to usual practice.
Hypothesis 2B. A documentation-based CDSS “smart form” will increase adherence with guidelines for the management of patients with CAD and ARI when compared to usual practice.
Aim 3. To assess the perceived value of EHR quality dashboards by clinicians and their marginal impact over smart forms on compliance with best practices in ARI and CAD.
Hypothesis 3A. An EMR-based “quality dashboard” system will provide additional benefit over documentation-based CDSS “smart form” in the management of patients with CAD and ARI.
Hypothesis 3B. Barriers to the effective use of computer-based quality improvement strategies can be identified.
Computer-based clinical decision-support systems (CDSS) are health information tools that combine education, physician participation, and feedback via reminders. These information technologies have the potential to change physician behavior at the precise time that clinical decisions are being made. However, such systems are still not used broadly and the full potential of CDSS remains to be tested. Moreover, when computerized reminder systems have resulted in demonstrable improvements, often this improvement has been less than anticipated.
Issues of usability and integration into the clinicians’ workflow are two most important barriers to the effectiveness of CDSS. One potential solution under development at Partners Healthcare is to integrate decision support into clinical documentation templates, thus facilitating clinical decision support, ordering, patient education, and documentation in a single step. We believe that Smart Forms (SF) have the potential to increase the perceived value and impact of EHRs for end-user physicians.
Direct feedback to physicians regarding the quality of care they provide has also been shown to be significant for improving guideline adherence. Documentation-based CDSS facilitates the acquisition of key quality data, which can then be presented in an efficient and concise manner in a Quality Dashboard (QD). In addition, such Quality Dashboards, linked to the electronic medical records, can enhance feedback by providing actionable, population-based information on quality of care, adherence to guidelines in relation to local and national benchmarks, and identify patients most in need of attention.
However, to date, very few EHRs have developed such features and functions. Of those that might have these tools, few institutions have taken advantage of these features or have systematically tested them.. We, therefore, designed, developed and implemented Smart Forms and Quality Dashboards in over 20 primary care practices in the Partners Health Care system with over 400 clinician study participants. These tools provided clinical decision support in three clinical areas – ARI, CAD, and diabetes. We then analyzed their effect on quality of care and adherence to guidelines using a randomized control trial strategy.
Subject Characteristics and Enrollment Study subjects included patients seen at a number of outpatient primary care practices associated with Brigham and Women’s and Massachusetts General Hospitals. Number of participating practices varied somewhat depending on the stage of the study, from 10 to 27. All attending and resident physicians in a given practice were included in the study. Patient population consisted of patients who made at least one visit to the clinic during the study period.
The study consisted of two cohorts, one for ARI and one for CAD. The CAD cohort of included patients with a diagnosis of coronary artery disease, younger than age 85, registered in the practice for at least one year, not living in a nursing home or with metastatic cancer. The ARI cohort included patients with a billing diagnosis of non-specific upper respiratory infection, otitis media, sinusitis, pharyngitis, acute bronchitis, pneumonia, or influenza. There were no age limits on the ARI cohort.
Procedures Prototypes of the Smart Forms and Quality Dashboards were developed and systematically tested with Partners physicians using a portable testing lab and the Questionnaire for User Interface Satisfaction (QUIS). Then, preliminary versions of the interventions were pilot tested in Partners practices for 6-8 weeks to prove feasibility, handle logistical issues, and conduct an informal “before-and-after” test of effectiveness. Feedback was used to optimize the interventions. Finally, a controlled trial, randomized by practice, was conducted in 27 Partners ambulatory primary care practices that use the LMR, comparing usual care, use of Smart Forms alone, and use of Smart Forms plus Quality Dashboards (See Table 1 for pilot and RCT dates).
Detailed usability surveys were given to physicians post-intervention to identify barriers to the use of these interventions. The data were also used to correlate different aspects of usability, actual use of the systems (measured by capturing screen navigation data), and their effectiveness in improving documentation and quality of care.
Physicians in the clinics randomized to the interventions were alerted to the introduction of the Smart Forms and Quality Dashboards through regular email announcements about LMR enhancements. Additionally, physicians in the intervention clinics were trained to use the Smart Forms in an in-person educational sessions at each clinic ~3 weeks prior to the beginning of the intervention period. Physicians in control clinics continued using an existing clinical decision support tool – End Of Visit (EOV) during the study period.