This is an open-access article distributed under the terms of the Creative Commons Attribution-NonCommercial-NoDerivatives License, which permits unrestricted reproduction and distribution, for non-commercial purposes only; and use and reproduction, but not distribution, of adapted material for non-commercial purposes only, provided the original work is properly cited.
Objective We identified the methods used and determined the roles of electronic health records (EHRs) in detecting and assessing adverse drug events (ADEs) in the ambulatory setting.
Methods We performed a systematic literature review by searching PubMed and Google Scholar for studies on ADEs detected in the ambulatory setting involving any EHR use published before June 2017. We extracted study characteristics from included studies related to ADE detection methods for analysis.
Results We identified 30 studies that evaluated ADEs in an ambulatory setting with an EHR. In 27 studies, EHRs were used only as the data source for ADE identification. In two studies, the EHR was used as both a data source and to deliver decision support to providers during order entry. In one study, the EHR was a source of data and generated patient safety reports that researchers used in the process of identifying ADEs. Methods of identification included manual chart review by trained nurses, pharmacists, and/or physicians; prescription review; computer monitors; electronic triggers; International Classification of Diseases codes; natural language processing of clinical notes; and patient phone calls and surveys. Seven studies provided examples of search phrases, laboratory values, and rules used to identify ADEs.
Conclusion The majority of studies examined used EHRs as sources of data for ADE detection. This retrospective approach is appropriate to measure incidence rates of ADEs but not adequate to detect preventable ADEs before patient harm occurs. New methods involving computer monitors and electronic triggers will enable researchers to catch preventable ADEs and take corrective action.
Keywords: review, drug-related side effects and adverse reactions, adverse drug reaction reporting systems, medication errors, medical records systems, computerized, ambulatory care
Adverse drug events (ADEs) are anticipated and unanticipated side effects of taking certain medications. 1 These events occur both in the inpatient and outpatient settings, often leading to patient injury or death. However, previous research has focused on occurrences of ADEs in the inpatient setting. More recent work has characterized rates in the ambulatory setting to be estimated at anywhere between 3 and 38%. 2 3 4 5 6
Reporting on the rates of occurrences has evolved with the implementation of electronic health records (EHRs) and the integration of computerized provider order entry (CPOE) with clinical decision support. Support for CPOE and its usefulness in reducing ADEs has been established in the inpatient setting. 7 Previously described ADE detection methods rely on the “gold standard” of manual chart review by physicians and other trained health professionals of medical notes, laboratory results, and changes made to prescriptions. This process takes considerable time and resources to perform on a large scale. Many ADEs that occur in the outpatient setting require further means of identification, especially if patients do not seek treatment for their symptoms. One approach to measuring ADEs that occur in the outpatient setting relies on patient surveys. 8 Recent studies have created tools such as electronic triggers and automated computer monitors to assist in detecting ADEs. 9 10 11 12 These methods either use the EHR as a data source or have incorporated those tools as built-in functionality.
More than 4 billion prescriptions are filled per year in the outpatient setting. 2 For this reason, there is a need to examine the outpatient setting where ADEs are often difficult to detect due to underreporting and lack of treatment. Previous reviews have assessed overall rates of occurrence, but none have performed a detailed comparison on the methods of identification, especially in recent years, during which EHRs may have changed the way ADEs are detected. In this study, we aimed to identify the methods used and determine the roles of EHRs in detecting and assessing ADEs in the ambulatory setting through systematic review of published literature.
We searched the PubMed database and Google Scholar for studies published before June 2017 on ADEs detected in the ambulatory setting involving some degree of use of an EHR. We also used references cited as an additional means of identifying studies. The Medical Subject Headings terms searched in PubMed were (medication errors or adverse drug reaction reporting systems or drug therapy/adverse effects or drug-related side effects and adverse reactions or iatrogenic disease/drug therapy) and (emergency medical services or primary health care or patient admission or hospitalization or outpatients or ambulatory care or ambulatory care facilities or physicians, family or family practice) and (medical records systems, computerized or medication systems or software or ambulatory care information systems or drug therapy, computer-assisted or medical order entry systems or decision support systems, clinical). The keywords searched in Google Scholar were (adverse drug event OR medication error) AND (ambulatory OR outpatient OR primary care) AND (electronic health record OR electronic medical record). We limited the Google Scholar search to the top 100 results.
All abstracts were independently screened by two reviewers; abstracts determined to be potentially relevant by either reviewer were included for full-text analysis. Both reviewers then independently reviewed the full-text articles for inclusion in the final analysis. Differences were discussed and reconciled between the authors. We included peer-reviewed articles published in English from any country as long as the study aimed to measure in the incidence of ADEs in an ambulatory setting and incorporated use of an EHR. Studies were excluded if they did not measure ADEs in an ambulatory setting, measured ADEs but without use of an EHR, were systematic reviews or meta-analyses, were not complete with data and results, or not accessible as full text. From the full-text articles we manually extracted: study setting, study design, sample size, follow-up time, ADE detection methods, EHR role, ADE definitions, ADE prevalence, and limitations.
We screened a total of 2,280 studies and reviewed 57 full-text articles. We included 30 studies in our final analysis. We excluded 2,250 studies that occurred in an inpatient setting, measured ADEs but without use of an EHR, were systematic reviews or meta-analyses, were not complete with data and results, or not accessible as full text ( Fig. 1 ). All included studies are listed in Supplementary Table S1 (available in the online version).
Flow diagram of included studies.
Eighteen studies were conducted in the United States, 1 in Australia, 3 in Canada, 2 in France, 1 in Germany, 1 in New Zealand, 2 in Spain, 1 in Sweden, and 1 in Taiwan. Five studies were conducted in hospitals, 24 in ambulatory care, and 1 took place at both settings. Four studies were randomized controlled trials, 9 were prospective, and 17 were retrospective studies.
Table 1 provides a summary of EHR use across included studies. In 27 studies, the EHR served as a data source for indications that an ADE had occurred. In two studies, the EHR was not only a source of data, but it also included clinical decision support that appeared while providers were ordering medications via CPOE. In the Genco et al 2016 study, the EHR was used as a source of data and had a role in analysis, as reports generated by the system were used by researchers to identify ADEs. 13
| Role of EHR | Number of studies | Citations |
|---|---|---|
| Source of data only | 27 | 2 9 14 15 16 17 18 19 21 23 24 26 27 29 30 31 32 36 37 38 39 40 41 42 43 44 45 |
| Source of data, decision support | 2 | 20 25 |
| Source of data, role in analysis | 1 | 13 |
Abbreviation: EHR, electronic health record.
All included studies used a combination of methods to identify and characterize ADEs in the ambulatory setting. Methods included manual chart review by trained nurses, pharmacists, and/or physicians; prescription review; computer monitors; electronic triggers; International Classification of Diseases (ICD) codes; natural language processing of clinical notes; and patient phone calls and surveys. Two studies report using return visit to the emergency department (ED) or admission from the ED within 24 hours as their ADE trigger. 14 15 None of the studies provided a comprehensive list of search phrases, laboratory values, or logic rules. Five studies utilized the Naranjo algorithm to determine the likelihood the ADE occurred as a result of a drug rather than other factors. 13 16 17 18 19 Two used the Beers Criteria in studies involving adults aged 65 years or older. 20 21
The most common role the EHR played was to act as a source of data for researchers to use to identify ADEs. EHRs took this passive role in 27 included studies. Instead of performing paper chart review, researchers searched through electronic charts for indicators of an ADE. Chart review was conducted by a combination of trained abstractors, research nurses, physicians, pharmacists, and toxicologists. In one study, the case file was reviewed by senior ED nurses, and if it was not rejected by them, the file was then passed over for review by emergency physicians. 15 In a French study, a committee of clinical pharmacologists, internists, and general practitioners conducted the case review. 22 In all cases, chart review was maintained as the “gold standard” of ADE identification, even in studies where a computer monitor or electronic trigger was used. 16
Studies included varied approaches to searching the EHR for signs of an ADE. Laboratory values, clinical notes, and ICD codes were commonly used. Cantor et al searched free-text notes for trigger phrases indicative of ADEs. 23 Brenner et al instead identified six laboratory values, international normalized ratio > 5, serum creatinine > 2.5, blood urea nitrogen > 60, alanine aminotransferase > 84, aspartate aminotransferase > 80, and thyroid-stimulating hormone undetectable while on levothyroxine, which were used to determine at what stage of the medication process the ADE occurred. 9 10 These laboratory values were adapted from a more comprehensive tool developed by Singh et al. 10 Only the laboratory values were used because they were shown to have a high positive predictive value, and researchers were able to extract the rest of the data associated with those values. Gandhi et al developed more sophisticated tools to search through laboratory values, medication lists, and applied logical rules to decide whether a potential ADE was present. This study also created a search monitor that worked off a set of rules to search the free-text electronic notes for symptom words that may have signaled an ADE. 24 A similar search tool was created by Honigman et al that looked through ICD-9 codes, allergy rules, computer event monitoring rules, and an automated chart review utilizing text searching of the EHR. 16
In three studies, EHR use was more innovative. 13 20 25 In two of these studies, CPOE provided decision support in the form of alerts and pop-up notifications. 20 25 In the Terrell et al study, clinical decision support was provided in a randomized controlled trial for nine medications that were determined to represent 80% of potentially inappropriate medications prescribed to seniors in the ED. The proportion of ED discharges that resulted in a potentially inappropriate medicated was 3.9% for the control group and 2.6% for the intervention group. 20 In the second study that included EHR as decision support, researchers only examined physicians' responses to dose-range alerts, but did not measure incidence of ADE occurrence due to prescription errors. 25 The third study by Genco et al utilized an EHR system that created data-based reports on patient safety, which were then included in the review process of identifying ADEs. 13
Commonly described limitations across studies were primarily related to the dependence on EHRs as a true source of information. A lack of standard documentation practices between physicians and across practices would not be representative of the observed rate of ADE occurrences; missing relevant information in patient charts may have led to misclassification, and errors not documented in the EHR may not have been identified, leading to an underreporting of ADEs. 23 26 27 28 29 In the case of Brenner et al looking at laboratory value triggers, it was unclear from EHR documentation whether errors occurred due to a lack of monitoring or appeared in the course of adhering to recommended medication monitoring. 9 Study designs often focused on catching prescribing error and would miss preventable ADEs, such as those due to wrong patients receiving medications, an incorrect diagnosis causing the wrong medication to be prescribed, or a drug–drug interaction with other home medications. 17 30 31
Studies also acknowledged that bias might have been introduced in cases where researchers and reviewing physicians were not blinded to the purpose of the study. Researchers might have been more careful in their reviews or excluded high-risk patients from the study, and providers may have been more careful when writing prescriptions. 25 31 32 In addition, Abramson et al noted that ADEs were best measured by chart review in combination with patient interviews or surveys, an approach that was not used in many of the included studies. 18
We identified 30 studies that met review specifications. We included studies that utilized EHRs in the study methods of identifying ADEs in the ambulatory setting. The majority of studies examined used EHRs as sources of data for chart review, replacing traditional paper chart review with electronic laboratory values and visit notes. Select studies highlight how researchers created computerized monitors and search tools that were able to comb through electronic patient data to identify ADEs.
EHR use has increased in recent years with the advancement of health information technology and Meaningful Use directives. 33 ADE research has spanned decades, and while prior studies have examined ADEs in the ambulatory setting, none have reported in detail the methods of identifying ADEs in the ambulatory setting with a focus on EHRs. Thomsen et al conducted a systematic review on incidences of ADEs in ambulatory care in 2007. While similar study search criteria were used, the previous review focused on characteristics of ADEs and not methods of identification, as in this study. 34 The ambulatory setting of our review highlights the unique challenge of outpatient identification of ADEs as compared with previous inpatient studies. Patients do not have close contact with their physicians, unlike in a hospital setting where physicians examine patients on a daily basis. They are responsible for obtaining and administering their own medications and do not maintain as thorough records as hospitals do, thus limiting the efficacy of retrospective chart review. 35 Examining methods of ADE identification is necessary to ensure researchers are obtaining the truest measure of ADEs in the ambulatory setting and that the methods can be reproduced in future research.
In this review, we found that EHRs were primarily used as sources of data for ADE detection. The majority of studies reviewed utilized a retrospective approach, which was useful to measure incidence rates of ADEs but not to catch preventable ADEs. Studies that created electronic tools capable of searching through the EHR to detect trigger phrases or laboratory values show promise of a transition away from complete manual chart review for the detection of ADEs. Manual chart review limits researchers and physicians to detecting ADEs after they have already happened. With computer monitors and electronic triggers searching the EHR in real time, health care providers may be able to identify preventable ADEs and take corrective action before patient harm occurs.
Future research is needed to measure consistency of documentation in various ambulatory settings, from large outpatient clinics to small primary care facilities. If there is a large disparity in patient data being documented, then study results will not be representative of the true measure of ambulatory ADEs.
This systematic review has some limitations. We searched the publicly available databases PubMed and Google Scholar, thus potentially excluding literature published in other sources. Inherent publication bias limited the number of articles available for review. Fewer studies were available in published literature on ADEs in the ambulatory setting. Because we were only interested in studies using EHRs, it limited us to looking at more recent studies, where EHR use became more prevalent. Due to the cost of EHRs, studies were most likely conducted at facilities that had the resources to afford an EHR, so results may not be generalizable.
Beyond the limitations of this review, EHRs are also limited by the information entered into them; a lack of standard documentation requirements may lead to incomplete charts that hinder ADE detection and monitoring. Medication reconciliation is another important step that needs to be completed during patient visits to ensure the med list is up-to-date in the system. Together with physician compliance, enhanced research tools that work with EHRs will enable researchers to better measure, characterize, and detect ADEs in the ambulatory setting.
ADEs are directly tied to patient outcomes and patient safety. Research in this area would benefit patients by improving ADE detection, providers by increasing their quality of care, and hospitals by decreasing ADE rates.
