Patient Safety Tip of the Week


March 3, 2009       Overriding Alerts…Like Surfin’ the Web


Was there an ad present when you came to this web page? The answer is “no” (because we don’t do any ads!) but the point is: you don’t remember whether there was one or not! The web has opened up vast resources to us but at the same time has shaped some of our behaviors. When we “surf” the web we are actually usually looking for a specific piece of information or performing a specific task. We have learned to ignore all the things that are extraneous to that task. If there is an ad or a request to fill out a survey, we click it off or simply ignore it or we just leave the site.


So should “alert fatigue” on computerized physician order entry (CPOE) come as a surprise to anyone? Just as you have developed “web ad fatigue” or staff in ICU’s have developed “alarm fatigue” where they tend to ignore some alarms because alarms are always going off, clinicians readily develop “alert fatigue” where they ignore all alerts and reminders that pop up on the computer screen.


While alert fatigue is not a new phenomenon, two recent papers highlight its widespread nature and the many issues related to it. Lo et al. (Lo et al. 2009) found that non-interruptive alerts recommending baseline lab testing during medication order entry in an ambulatory setting were not effective in improving test ordering. Isaac et al. (Isaac et al. 2009) found that clinicians using an outpatient e-prescribing system accepted only 9.2% of drug interaction alerts and only 23% of allergy alerts.


That certainly questions the likelihood of quick success for the CMS e-prescribing initiative or the approximately $20 billion in President Obama’s stimulus package that is targeted for healthcare information technology. That $20 billion investment was estimated to generate a downstream savings of many billion of dollars due to reduction of medical errors and better efficiencies. We remain strong advocates of using IT to improve quality and patient safety and save money in doing so. However, the dismal results of the above two studies show we have a lot of work to do to reap those benefits.


So what can we learn from these studies? In the Isaac study there was no significant difference in alert acceptance rates across different specialties. Clinicians who wrote more prescriptions were less likely to accept the drug interaction alerts, though those who used the e-prescribing system longer were somewhat more likely to accept alerts. Importantly, clinicians were only slightly more likely to accept high-severity alerts than moderate- or low-severity ones. Especially if a patient had previously received an alerted medication the physician was likely to override the alert (you’ll recall from many of our previous columns that alerts often lead to a reduction in new prescriptions for a medication but that physicians seldom stop a medication the patient is already taking).


Contrast that to a study done on inpatients (Paterno et al. 2009)  which showed that tiering of drug-drug interaction (DDI) alerts by severity level was successful at improving compliance rates. In that study, acceptance rates for DDI’s were compared at two comparable academic hospitals within a large system. Both used the same DDI database. At one hospital, the alerts were tiered by severity. Level I alerts (the most serious) accounted for only 0.2% of alerts but required a “hard stop” (i.e. the ordering physician could not continue with the order). At the control (non-tiered) hospital, where the clinician was not interrupted in any way, only 34% of those same alerts were accepted. For the next most serious level II alerts (which required at least some action by the clinician at the tiered hospital), the acceptance rates were also more likely to be accepted at the tiered hospital (29% vs. 10%). So at least on the inpatient side, it is clear that hard stop alerts or interruptive alerts are more likely to have an impact.


So if you have a patient safety issue you are trying to address through CPOE, your chance of influencing it with non-interruptive (“informational”) alerts is not very good no matter how colorful or animated you make that alert (remember, you also easily ignore those animated dancing figures in the webpage ads!). For example, if you are attempting to reduce the inappropriate use of Foley catheters, simply listing the legitimate indications for a Foley is not likely to reduce their use. But if you require the ordering physician to input a reason for the Foley by checking a checkbox with those legitimate reasons or inputting free text before they can proceed, you are much more likely to have an impact.


Another consideration is use of reflex orders. For example, if you don’t want patients on full anticoagulation going unmonitored consider automatically ordering the lab monitoring tests when the anticoagulant is initially ordered. Obviously, such reflex ordering needs to be done under a protocol approved by your medical staff.


And, whenever possible, use standardized order sets to accomplish your goals rather than alerts and reminders. Standardized order sets work well but keep in mind that only a fraction of orders during a hospitalization are entered at a time when such order sets are likely to be used (eg. on admission or postoperatively).


A study from the same health system as the Paterno study (Shah et al. 2006) also demonstrated a beneficial effect of tiered alerts on the outpatient side. 67% of interruptive Level 1 or 2 alerts were accepted in this study. The knowledge base used in this study to generate alerts probably resulted in more clinically focused alerts than those generated in the commercial knowledge base used in the Isaac study. The Shah study also details many of the reasons clinicians chose to override alerts, providing keen insight into future development of alerts.


Your job does not stop when you deploy new rules, alerts and reminders. You need to have in place a system that tracks the number of times an alert is triggered, how often it is accepted or overridden, what the reasons are for overrides, what sort of responses clinicians made to alerts, and whether the alert achieved the clinical outcomes you were attempting to achieve (or avoid), getting both objective and subjective measures of the success or failure of that rule/alert. That review must take place early and often (we recommend at one and three months). Rules/alerts with high override rates either need to be removed or reconsidered. In addition, capturing the reasons for overrides may help design of future alerts so that such reasons can be included in dropdown boxes or checklists for overrides.


Providing the clinician with alternative actions when an alert is presented may be helpful, though there is not a lot of evidence to validate that. For example, in the study by Lo et al. the physician sometimes had to use a different modality to order the monitoring lab test. It is conceivable that such disruption in workflow may have played a significant role in failure to follow the alerts. While you want to minimize interruptions in the usual workflow for your clinicians, you also want to make it easy for them to do the right thing. If they cannot easily accept the alert and do the correct thing with one click (or a few at most), they are probably going to ignore the alert. It’s like designing a website – too many clicks and you lose the surfer.


Everyone agrees that the fewer interruptions you cause for physicians, the more they are likely to adopt CPOE. So you need to put your stake in the ground – pick a relatively small number of serious things you are trying to prevent and use more interruptive techniques to discourage those things. The key question is whether the non-interruptive alerts and reminders are ever of value. We spend a great deal of time developing many of those despite lack of good evidence that they actually change outcomes. They are the ones that are like the internet ads – we’ll bet you never pay attention to them either!






Lo HG, Matheny ME, Seger DL, Bates DW, Gandhi TK. Impact of non-interruptive medication laboratory monitoring alerts in ambulatory care. J Am Med Inform Assoc. 2009; 16:66-71



Isaac, Thomas MD, MBA, MPH; Weissman, Joel S. PhD; Davis, Roger B. ScD; Massagli, Michael PhD; Cyrulik, Adrienne MPH; Sands, Daniel Z. MD, MPH; Weingart, Saul N. MD, PhD Overrides of Medication Alerts in Ambulatory Care. Archives of Internal Medicine. 169(3):305-311, February 9, 2009.


Paterno MD, Maviglia SM, Gorman PN, et al. Tiering Drug–Drug Interaction Alerts by Severity Increases Compliance Rates. J. Am. Med. Inform. Assoc. 2009; 16(1):40-46. PrePrint published January 1, 2009; doi:10.1197/jamia.M2808,HWELTR



Shah NR, Seger AC, Seger DL, et al. Improving Acceptance of Computerized Prescribing Alerts in Ambulatory Care. J. Am. Med. Inform. Assoc. 2006; 13(1):5-11. PrePrint published January 1, 2006; doi:10.1197/jamia.M1868,HWELTR









Patient Safety Tip of the Week Archive


What’s New in the Patient Safety World Archive