Patient Safety Tip of the Week
September 20, 2011
When Practice Changes the Evidence: The CKD Story
In our January 2011 What’s New in the Patient Safety World column “No Improvement in Patient Safety: Why Not?” we focused on our failure to change the culture as the main reason we have not moved the bar much in the patient safety movement. However, we also noted that we have ourselves to blame in that many of the patient safety interventions we have implemented have turned out to adversely affect patient outcomes. In retrospect, the evidence for some of these interventions was “soft”. We hyped perioperative beta blockers for just about anyone undergoing surgery. Now we’ve learned that they may actually increase mortality. We overdid it on prophylactic use of agents for gastric acid suppression. Now we realize they may have played a role in development of C. difficile infections and may even have increased the risk for ventilator-associated pneumonia (VAP), the very condition for which they were commonly being used. We made antibiotics within 4 hours for community-acquired pneumonia a quality and pay-for-performance standard, only to see many patients who turned out not to have pneumonia being unnecessarily treated with antibiotics. Even SCIP (Surgical Care Improvement Project) showed little impact of adherence to individual practices on patient outcomes (see our August 2010 What’s New in the Patient Safety World column “SCIP: Disappointing Outcomes on SSI’s. What’s Next?”).
And there are other quality improvement initiatives we undertook based on a good evidence base that have failed to produce the desired outcomes because changes in practice changed the evidence. The story of chronic kidney disease (CKD) illustrates this well. Back in the early 2000’s the evidence base was convincing that, among patients who eventually went on dialysis, those who had late referral to a nephrologist had substantially increased mortality and costs in the first year on dialysis. So we all made concerted efforts to identify patients with progressing CKD and get them to nephrologists earlier. In Western New York we had a coalition of physicians (in both primary care and nephrology), hospitals, consumer advocacy group, payors and the business community that addressed the issue. We became the first community to get all laboratories to use the MDRD equation to provide an estimated glomerular filtration rate (eGFR) each time they reported a serum creatinine. And we developed tools to help PCP’s better manage CKD at each stage and facilitate early referral to the nephrologists. We clearly succeeded in increasing the number of CKD patients getting early referral, as the rest of the country has now also done. The number of CKD patients who see a nephrologist at least a year before starting dialysis nationwide increased from about 30% in 1996 to almost 50% in 2006 (Winkelmayer 2011).
But we made a fundamental mistake in our community CKD collaboarative. Those of you involved in quality improvement who use the model espoused in The Improvement Guide (Langley 1996) know that the key question after you identify something you want to change is “How will we know that the change is an improvement?”, i.e. how will we measure the effects (both intended and unintended) of our efforts. We certainly had a plan and identified sources for measurement of the surrogate measures (like % with eGFR reported and % of patients with early nephrology referral). But our ultimate goals were to (1) prevent or delay the need for dialysis in CKD patients (2) reduce the mortality of those patients who do need dialysis (3) improve quality of life in the CKD population and (4) reduce overall costs of managing the CKD population. Basically, these were “Triple Aim” goals. The measures for all the above were longer term outcome measures and we simply did not have the infrastructure in place at the time to ensure that we’d adequately measure these more important outcomes down the line.
But now we know what the outcome likely was. A new study (Winkelmayer 2011), using national data from the United States Renal Data System and Medicare claims, shows that early referral to nephrologists has resulted in essentially no improvement in survival rates for patients one year after initiation of dialysis.
So what happened? How could the previous evidence on early vs. late referral have been so wrong? The answer is that it was probably not wrong at the time. What happened was that early referral coincided with (and perhaps inadvertently caused) some changes in clinical management of CKD patients.
Two significant trends were noted in the Winkelmayer study over the 10-year period. One was that dialysis was increasingly initiated at higher GFR levels (i.e. the patients had more preserved renal function when dialysis was begun). And there was a substantial increase in the usage of erythropoiesis-stimulating agents (ESA’s). You are all aware of the recent attention to the detrimental effects of aggressive erythropoiesis-stimulating agent use (the CREATE, CHOIR, and TREAT studies). Winkelmayer et al. also note that recent studies have also questioned the efficacy of several other interventions, such as aggressive blood pressure lowering, renal artery revascularization, statin use, and management of metabolic bone disease in CKD.
There has been a striking trend over the last decade or so of earlier initiation of dialysis (that is, dialysis is being started in patients with higher eGFR levels). The recently completed IDEAL trial (Cooper 2010) showed no benefit from earlier initiation of dialysis and another recent analysis of national data (Rosansky 2011) actually suggests that early initiation of dialysis may be harmful. And another study (Tamura 2009) showed a substantial and sustained decline in functional status in nursing home patients initiated on dialysis.
Moreover, the cost effectiveness analysis accompanying the IDEAL trial (Harris 2011) showed that early initiation of dialysis resulted in higher costs (almost $16,000 higher) and did not improve quality of life.
In the previous literature, the prime driver of both mortality/morbidity and cost in patients during the first year of dialysis was that dialysis was often started emergently. The patient was often admitted to a hospital for an intercurrent medical or surgical problem and the renal status deteriorated to the point that dialysis was necessary. That dialysis thus had to begin when the patient had suboptimal vascular access for dialysis (i.e. they did not have a mature fistula what would be used for ongoing dialysis).
But one wonders if even such well-intentioned programs like the “Fistula First” program may have played a role in the changes in practice. Clearly, having a mature fistula in place when dialysis is initiated is beneficial. However, one wonders whether having those fistulas ready may also play a role in the trend toward initiating dialysis in patients with more preserved renal function.
But you have to look at root causes to see how things can get out of hand. Many of the root causes in the CKD story are related to conflicts of interest at multiple levels. A commentary (Senekjian 2011) noted that there are numerous financial incentives for nephrologists in the US to initiate dialysis early. In addition, the use of erythropoiesis-stimulating agents (ESA’s) was also impacted by conflicts of interest. Of course, pharmaceutical companies benefit from their use. But in most cases so do the providers who prescribe them. Even some of the disease management companies that focused on CKD used their purchasing clout to get better pricing on ESA’s that they passed on to their clients. But in doing so, they probably contributed to the aggressive use of ESA’s that, retrospectively, may have contributed to harmful outcomes in some patients. And one needs to look at the process of guideline development and the means of funding the sponsoring group(s). You’ll find that, even though they may be “consumer advocacy” groups, they derive much of their funding from “industry”. We found it most interesting when the consumer advocacy group in the Western New York collaborative volunteered to help put together the educational materials for PCP’s to help manage CKD. Chapter One of the proposed materials was “Management of Anemia”!
So are there lessons learned here? Yes, there are several:
· Ensure you will measure the important patient outcomes rather than surrogate measures in any quality improvement or patient safety project.
· Always be vigilant for unintended consequences of you actions.
· Be particularly wary of perverse incentives and conflicts of interest that may come into play.
· Recognize that even practices that are “evidence-based” when you start, they may be changed by changes that take place in practice.
· Don’t be so anxious to get your project going that you ignore all the above lessons!
References:
Winkelmayer WC, Liu J, Chertow GM, Tamura MK. Predialysis Nephrology Care of Older Patients Approaching End-stage Renal Disease. Arch Intern Med. 2011; 171(15): 1371-1378
http://archinte.ama-assn.org/cgi/content/abstract/171/15/1371
Langley GJ, Nolan KM, Nolan TW, Norman CL, Provost LP, The Improvement Guide. A Practical Approach to Enhancing Organizational Performance. San Francisco: Jossey-Bass Publishers, 1996
http://www.amazon.com/Improvement-Guide-Organizational-Performance-Jossey-Bass/dp/0787902578
Drüeke TB, Locatelli F, Clyne N; et al, CREATE Investigators. Normalization of hemoglobin level in patients with chronic kidney disease and anemia. N Engl J Med. 2006; 355(20): 2071-2084
http://www.nejm.org/doi/pdf/10.1056/NEJMoa062276
Singh AK, Szczech L, Tang KL; et al, CHOIR Investigators. Correction of anemia with epoetin alfa in chronic kidney disease. N Engl J Med. 2006; 355(20): 2085-2098
http://www.nejm.org/doi/pdf/10.1056/NEJMoa065485
Pfeffer MA, Burdmann EA, Chen C-Y; et al, TREAT Investigators. A trial of darbepoetin alfa in type 2 diabetes and chronic kidney disease. N Engl J Med. 2009; 361(21): 2019-2032
http://www.nejm.org/doi/pdf/10.1056/NEJMoa0907845
Cooper BA, Branley
P, Bulfone L, et al. A Randomized, Controlled Trial of Early versus Late
Initiation of Dialysis. N Engl J Med 2010; 363: 609-619
http://www.nejm.org/doi/pdf/10.1056/NEJMoa1000552
Rosansky SJ, Eggers
P, Jackson K, et al. Early Start of
Hemodialysis May Be Harmful
Arch Intern Med, Mar 2011; 171: 396 – 403
Tamura MK, Covinsky
KE, Chertow GM, et al. Functional Status of Elderly Adults before and after
Initiation of Dialysis. N Engl J Med 2009; 361: 1539-1547
http://www.nejm.org/doi/pdf/10.1056/NEJMoa0904655
Harris A, Cooper BA,
Li JJ, et al. Cost-Effectiveness of Initiating Dialysis Early: A Randomized
Controlled Trial. Am J Kidney Dis 2011; 57(5): 707-715
http://www.ajkd.org/article/S0272-6386%2811%2900049-7/abstract
Senekjian HO. Regarding Early Initiation of Dialysis. Am J Kidney Dis 2011; 58(3): 492
http://www.ajkd.org/article/S0272-6386%2811%2901040-7/fulltext
http://www.patientsafetysolutions.com/
What’s New in the Patient Safety World Archive