Archived Comments for:
The Infectious Diseases Society of America Lyme guidelines: a cautionary tale about the development of clinical practice guidelines
Clinical practice guidelines need to be the best medicine
lou overman, none
14 June 2010
If professional societies like the IDSA can continue to use flawed guidelines, even when presented with irrefutable evidence that the guidelines are not good medicine, then this will result in a decline in public confidence in medicine as a whole. The public expects and deserves high standards.
No profession is above question, and bad behavior by some needs some outside enforcement action. Where is that action going to come from? Do we have to have class action lawsuits to get honest medicine?
Competing interests
None declared
IDSA Response to Stricker and Johnson
Diana Olson, Infectious Diseases Society of America
1 July 2010
The Infectious Diseases Society of America's primary concern is to enhance the care of children and adults based on sound scientific evidence. IDSA develops treatment guidelines through an extensive, thoughtful review of all of the evidence of a disease or condition, and its treatment options. We have published more than 60 practice guidelines for a range of infectious diseases, and our guidelines for Lyme disease are the only ones that have been challenged. Those guidelines were found to be valid by an extraordinary independent review, the results of which were made public in April 2010.
It is notable that Dr. Stricker and Ms. Johnson choose to criticize IDSA and its Lyme disease guidelines, rather than provide solid, scientific evidence that the treatments they espouse are valid. The reasons are clear. As yet, there is no scientific evidence that long-term antibiotics, often delivered intravenously for months or years, are beneficial for patients with “chronic” Lyme disease. There is, however, solid and abundant evidence that these treatments can be harmful, potentially leading to fatal infections, serious drug reactions, and the fostering of the development of life-threatening drug-resistant superbugs.
The IDSA’s full response to Johnson and Stricker is in process and a further comment will be uploaded giving the article details in due course.
Richard J. Whitley, MD, FIDSA President, Infectious Diseases Society of America
Competing interests
None
Insufficient Evidence and Poor Outcomes: IDSA Treatment Recommendations Rightly Ignored
Elizabeth Maloney, None
10 July 2010
Comments from IDSA president, Dr. Richard J. Whitley, suggest that he fully believes that “the best defense is a good offense”. Instead of addressing the shortcomings of the IDSA guidelines on Lyme disease, of which there are many, he assails Dr. Stricker and Ms. Johnson for not providing evidence that long-term treatments are valid.[1] He also tries to distract readers from considering the weak scientific underpinnings of the IDSA guidelines by raising the specter of unending courses of IV antibiotics and “life-threatening drug-resistant superbugs”; readers should not be fooled by such tactics.
Organizations which create treatment guidelines are obligated to prove the validity of their recommendations. This requires them to disclose the strength of their evidence so clinicians can use this information to judge the merits of the treatment recommendations. Because the IDSA Lyme guidelines issued 72 graded recommendations, it is easy for clinicians to lose sight of the fact that 54% of these, including 17 strong recommendations, were based on panel opinion.[2] Other guidelines developers, such as the American Academy of Pediatrics, require that the strength of a recommendation be matched to the strength of the underlying evidence;[3] unlike the IDSA, AAP would not restrict treatment options, via strong recommendations, purely on the basis of panel opinion.
The evidence strength ratings assigned by guidelines panels must be justifiable; even the pedestrian, IDSA-chosen review panel recognized that the strength of the supporting evidence had been stretched to reach the single-dose doxycycline prophylaxis recommendation.[4] And, evaluating the strength of an individual study requires more than a casual glance at the abstract and conclusion. When the article in question is written by a panelist on the 2006 guidelines, the examination should be especially vigorous so as to withstand charges of professional cronyism. This is also true when recommendations are issued to address areas of medical controversy.
This clearly did not happen with the IDSA guidelines. Consider the issue of treatment duration for erythema migrans, a contentious topic. The IDSA guidelines panel cited 8 prospective studies to support its recommendation; of these, only 2 investigated doxycycline regimens employing brief, 10 day treatment durations. In the study by Mazzarotti et al, the authors claimed the 10-day doxycycline arm had a 95% success rate.[5] However, of the 22 patients randomized to and completing this treatment, 7 were immediately retreated with doxycycline or amoxicillin and another patient later required IV ceftriaxone. Thus, 10 days of doxycycline failed to cure 36% of the patients, not 5%. One would think that such a gross overstatement of treatment success would have been caught by a diligent guideline panel; panelist Steere, as one of Mazzarotti’s co-authors, may have been best positioned to prevent the inclusion of this study in the guidelines. The other study, by guidelines panelist Wormser, had excessive drop-out rates.[6] At the study’s completion 49% of the subjects were “unevaluable”; at the earlier 12 month evaluation, 29% of the patients were already “unevaluable.” Biostatisticians warn against drawing outcome conclusions when drop-out rates exceed 20%;[7] thus, the panel also erred in citing the study by Wormser as supportive. If these studies are representative of what the IDSA considers “sound scientific evidence”, perhaps it is premature to be making recommendations in the first place.
After discovering a lack of support for the 10 day doxycycline regimen, I re-evaluated the data from the other 6 trials cited as supportive evidence for the early Lyme disease treatment recommendations.[8-13] During that process, I reanalyzed the outcome data using intent-to-treat methodology (ITT) as opposed to the complete-case(cc) or last-observation-carried-forward (LOCF) methods used in the original papers. ITT is the method preferred because CC and LOCF overstate treatment outcomes.[14] Differences in study designs and in the definitions of treatment success, improvement and failure make direct comparisons difficult but if success is defined as a return to the pre-morbid baseline without relapse during the observation period, then the overall success rates for doxycycline, amoxicillin and cefuroxime are roughly 65%. While this may seem incredulous to many, the review panel, which received my analysis in the course of its deliberations, suggested that future guidelines describe the first-line agents as “effective” rather than “highly effective”.[4]
Dr. Whitley expressed concerns regarding the use of long-term antibiotics in patients with persistent symptoms. There can be no doubt that such approaches carry risks but those risks must be weighed in light of the situation for which they are employed; this is not a case of using sledgehammer to swat a fly. The disease burden in this group is quite high, as the retreatment trials demonstrated.[15-17]
The IDSA guidelines also prohibit retreatment for patients with late neurologic Lyme disease who remain symptomatic following 30 days of ceftriaxone. This restriction is based on scant evidence. The guidelines cite only 4 trials, with a total of 96 patients representing a limited disease spectrum, which can be analyzed in terms of neurologic outcomes.[18-21] In this very small cohort, treatment successfully restored health in only 7 – 35% of the patients. Such a poor outcome is unacceptable for a patient group burdened with a disease causing a profoundly negative impact on the quality of their lives.
While physicians are cautioned to do no harm, it is clear that for the majority of patients with late neurologic Lyme disease, doing nothing more is harmful. To appease those looking for a scientific basis for additional antibiotic therapy, I suggest they read the 1999 study by Logigian et al.[21] In that open label trial using 30 days of ceftriaxone, one patient (who was well at the 6 month evaluation) reported a relapse, supported by a deterioration in his verbal and visual memory, 2 months later. Based on that information, the authors retreated him with 30 additional days of ceftriaxone and he demonstrated sustained improvement. Given that Steere served on the original guidelines panel and co-authored this paper, it is curious that the IDSA recommends against retreatment. Given the poor outcomes to shorter treatment durations and the disease burden, it is unconscionable.
Similarly detailed critiques can be made for the other major recommendations. Rather than shoot the messengers (Dr. Stricker and Ms. Johnson), Dr. Whitley should heed the message: the IDSA failed, in its initial and review efforts, to create impartial, conflict-free, evidence-based guidelines. Moreover, the errors of the guidelines panel were compounded by the review panel, which had an obligation to provide an unbiased review and right these transparent errors. Those of us who understand the situation lack mechanisms to resolve it. The duty remains with the IDSA members; physicians, heal thyselves.
Phyllis Mervine, California Lyme Disease Association
17 July 2010
The Infectious Diseases Society of America (IDSA) Lyme guidelines place commercial interests above quality of patient care, leaving seriously ill patients without effective treatment options. The narrow IDSA diagnostic criteria miss half the cases of Lyme disease, and up to half or more of the patients treated according to these guidelines remain ill. This is not acceptable.
The IDSA guidelines understate the seriousness of chronic Lyme disease, comparing ongoing Lyme symptoms to the "aches and pains of daily living." In contrast, peer-reviewed published studies show that people with chronic Lyme disease have disability and pain comparable to that of people with congestive heart failure and post-surgical pain, respectively. Schoolchildren with Lyme disease are classified as disabled and qualify for accommodations under Section 504 of the Americans with Disabilities Act. Deaths related to Lyme disease have also been reported.
The IDSA guidelines overstate the risks of long-term antibiotic treatment. Long-term treatment under the care of a qualified health care professional has been found to be safe and effective for a number of infectious diseases. The guidelines limit clinical judgment and hold physicians to arbitrary and unproven formulas. The IDSA-endorsed practice of terminating treatment despite persisting symptoms may result in advanced neurological injury, disability and death and constitutes medical negligence.
The California Lyme Disease Association (CALDA) recommends that patients with Lyme disease see a doctor affiliated with the International Lyme and Associated Diseases Society, who will individualize their treatment plan and consider patient preferences. Patients have the right to make informed choices based on their own personal values, just like patients with cancer or other diseases. In our opinion anything less is immoral, illegal, and inhumane.
Phyllis Mervine, EdM President, CALDA Editor-in-Chief, Lyme Times
Clinical practice guidelines need to be the best medicine
14 June 2010
If professional societies like the IDSA can continue to use flawed guidelines, even when presented with irrefutable evidence that the guidelines are not good medicine, then this will result in a decline in public confidence in medicine as a whole. The public expects and deserves high standards.
No profession is above question, and bad behavior by some needs some outside enforcement action. Where is that action going to come from? Do we have to have class action lawsuits to get honest medicine?
Competing interests
None declared
IDSA Response to Stricker and Johnson
1 July 2010
The Infectious Diseases Society of America's primary concern is to enhance the care of children and adults based on sound scientific evidence. IDSA develops treatment guidelines through an extensive, thoughtful review of all of the evidence of a disease or condition, and its treatment options. We have published more than 60 practice guidelines for a range of infectious diseases, and our guidelines for Lyme disease are the only ones that have been challenged. Those guidelines were found to be valid by an extraordinary independent review, the results of which were made public in April 2010.
It is notable that Dr. Stricker and Ms. Johnson choose to criticize IDSA and its Lyme disease guidelines, rather than provide solid, scientific evidence that the treatments they espouse are valid. The reasons are clear. As yet, there is no scientific evidence that long-term antibiotics, often delivered intravenously for months or years, are beneficial for patients with “chronic” Lyme disease. There is, however, solid and abundant evidence that these treatments can be harmful, potentially leading to fatal infections, serious drug reactions, and the fostering of the development of life-threatening drug-resistant superbugs.
The IDSA’s full response to Johnson and Stricker is in process and a further comment will be uploaded giving the article details in due course.
Richard J. Whitley, MD, FIDSA
President, Infectious Diseases Society of America
Competing interests
None
Insufficient Evidence and Poor Outcomes: IDSA Treatment Recommendations Rightly Ignored
10 July 2010
Comments from IDSA president, Dr. Richard J. Whitley, suggest that he fully believes that “the best defense is a good offense”. Instead of addressing the shortcomings of the IDSA guidelines on Lyme disease, of which there are many, he assails Dr. Stricker and Ms. Johnson for not providing evidence that long-term treatments are valid.[1] He also tries to distract readers from considering the weak scientific underpinnings of the IDSA guidelines by raising the specter of unending courses of IV antibiotics and “life-threatening drug-resistant superbugs”; readers should not be fooled by such tactics.
Organizations which create treatment guidelines are obligated to prove the validity of their recommendations. This requires them to disclose the strength of their evidence so clinicians can use this information to judge the merits of the treatment recommendations. Because the IDSA Lyme guidelines issued 72 graded recommendations, it is easy for clinicians to lose sight of the fact that 54% of these, including 17 strong recommendations, were based on panel opinion.[2] Other guidelines developers, such as the American Academy of Pediatrics, require that the strength of a recommendation be matched to the strength of the underlying evidence;[3] unlike the IDSA, AAP would not restrict treatment options, via strong recommendations, purely on the basis of panel opinion.
The evidence strength ratings assigned by guidelines panels must be justifiable; even the pedestrian, IDSA-chosen review panel recognized that the strength of the supporting evidence had been stretched to reach the single-dose doxycycline prophylaxis recommendation.[4] And, evaluating the strength of an individual study requires more than a casual glance at the abstract and conclusion. When the article in question is written by a panelist on the 2006 guidelines, the examination should be especially vigorous so as to withstand charges of professional cronyism. This is also true when recommendations are issued to address areas of medical controversy.
This clearly did not happen with the IDSA guidelines. Consider the issue of treatment duration for erythema migrans, a contentious topic. The IDSA guidelines panel cited 8 prospective studies to support its recommendation; of these, only 2 investigated doxycycline regimens employing brief, 10 day treatment durations. In the study by Mazzarotti et al, the authors claimed the 10-day doxycycline arm had a 95% success rate.[5] However, of the 22 patients randomized to and completing this treatment, 7 were immediately retreated with doxycycline or amoxicillin and another patient later required IV ceftriaxone. Thus, 10 days of doxycycline failed to cure 36% of the patients, not 5%. One would think that such a gross overstatement of treatment success would have been caught by a diligent guideline panel; panelist Steere, as one of Mazzarotti’s co-authors, may have been best positioned to prevent the inclusion of this study in the guidelines. The other study, by guidelines panelist Wormser, had excessive drop-out rates.[6] At the study’s completion 49% of the subjects were “unevaluable”; at the earlier 12 month evaluation, 29% of the patients were already “unevaluable.” Biostatisticians warn against drawing outcome conclusions when drop-out rates exceed 20%;[7] thus, the panel also erred in citing the study by Wormser as supportive. If these studies are representative of what the IDSA considers “sound scientific evidence”, perhaps it is premature to be making recommendations in the first place.
After discovering a lack of support for the 10 day doxycycline regimen, I re-evaluated the data from the other 6 trials cited as supportive evidence for the early Lyme disease treatment recommendations.[8-13] During that process, I reanalyzed the outcome data using intent-to-treat methodology (ITT) as opposed to the complete-case(cc) or last-observation-carried-forward (LOCF) methods used in the original papers. ITT is the method preferred because CC and LOCF overstate treatment outcomes.[14] Differences in study designs and in the definitions of treatment success, improvement and failure make direct comparisons difficult but if success is defined as a return to the pre-morbid baseline without relapse during the observation period, then the overall success rates for doxycycline, amoxicillin and cefuroxime are roughly 65%. While this may seem incredulous to many, the review panel, which received my analysis in the course of its deliberations, suggested that future guidelines describe the first-line agents as “effective” rather than “highly effective”.[4]
Dr. Whitley expressed concerns regarding the use of long-term antibiotics in patients with persistent symptoms. There can be no doubt that such approaches carry risks but those risks must be weighed in light of the situation for which they are employed; this is not a case of using sledgehammer to swat a fly. The disease burden in this group is quite high, as the retreatment trials demonstrated.[15-17]
The IDSA guidelines also prohibit retreatment for patients with late neurologic Lyme disease who remain symptomatic following 30 days of ceftriaxone. This restriction is based on scant evidence. The guidelines cite only 4 trials, with a total of 96 patients representing a limited disease spectrum, which can be analyzed in terms of neurologic outcomes.[18-21] In this very small cohort, treatment successfully restored health in only 7 – 35% of the patients. Such a poor outcome is unacceptable for a patient group burdened with a disease causing a profoundly negative impact on the quality of their lives.
While physicians are cautioned to do no harm, it is clear that for the majority of patients with late neurologic Lyme disease, doing nothing more is harmful. To appease those looking for a scientific basis for additional antibiotic therapy, I suggest they read the 1999 study by Logigian et al.[21] In that open label trial using 30 days of ceftriaxone, one patient (who was well at the 6 month evaluation) reported a relapse, supported by a deterioration in his verbal and visual memory, 2 months later. Based on that information, the authors retreated him with 30 additional days of ceftriaxone and he demonstrated sustained improvement. Given that Steere served on the original guidelines panel and co-authored this paper, it is curious that the IDSA recommends against retreatment. Given the poor outcomes to shorter treatment durations and the disease burden, it is unconscionable.
Similarly detailed critiques can be made for the other major recommendations. Rather than shoot the messengers (Dr. Stricker and Ms. Johnson), Dr. Whitley should heed the message: the IDSA failed, in its initial and review efforts, to create impartial, conflict-free, evidence-based guidelines. Moreover, the errors of the guidelines panel were compounded by the review panel, which had an obligation to provide an unbiased review and right these transparent errors. Those of us who understand the situation lack mechanisms to resolve it. The duty remains with the IDSA members; physicians, heal thyselves.
References
1. Whitley RJ. IDSA Response to Stricker and Johnson. http://www.peh-med.com/content/5/1/9/comments
2. Wormser GP, Dattwyler RJ, Shapiro ED, Halperin JJ, Steere AC, Klempner MS, Krause PJ, Bakken JS, Strle F, Stanek G, Bockenstedt L, Fish D, Dumler JS, Nadelman RB. The clinical Assessment, Treatment, and Prevention of Lyme Disease, Human Granulocytic Anaplasmosis, and Babesiosis: Clinical Practice Guidelines by the Infectious Diseases Society of America. Clin Infect Dis 2006;43(9):1089-134.
3. American Academy of Pediatrics: Steering Committee on Quality Improvement and Management. Classifying Recommendations for Clinical Practice Guidelines. Pediatrics 2004;114;874-877.
4. Infectious Diseases Society of America: Final Report of the Lyme Disease Review Panel of the Infectious Diseases Society of America. April 22, 2010. http://www.idsociety.org/Content.aspx?id=16499
5. Massarotti EM, Luger SW, Rahn DW, et al.. Treatment of early Lyme disease. Am J Med 1992; 92:396–403.
6. Wormser GP, Ramanathan R, Nowakowski J, et al.. Duration of antibiotic therapy for early Lyme disease: a randomized, double-blind, placebo-controlled trial. Ann Intern Med 2003; 138:697–704.
7. Schulz K, Grimes D. Sample size slippages in randomised trials: exclusions and the lost and wayward. Lancet 2002; 359: 781–85.
8. Luft BJ, Dattwyler RJ, Johnson RC, et al.. Azithromycin compared with amoxicillin in the treatment of erythema migrans: a double blind, randomized, controlled trial. Ann Intern Med 1996; 124:785–91.
9. Dattwyler RJ, Volkman DJ, Conaty SM, Platkin SP, Luft BJ. Amoxicillin plus probenecid versus doxycycline for treatment of erythema migrans borreliosis. Lancet 1990; 336:1404–6.
10. Eppes SC, Childs JA. Comparative study of cefuroxime axetil versus amoxicillin in children with early Lyme disease. Pediatrics 2002; 109:1173–7.
11. Nadelman RB, Luger SW, Frank E, et al.. Comparison of cefuroxime axetil and doxycycline in the treatment of early Lyme disease. Ann Intern Med 1992; 117:273–80.
12. Luger SW, Paparone P, Wormser GP, et al.. Comparison of cefuroxime axetil and doxycycline in treatment of patients with early Lyme disease associated with erythema migrans. Antimicrob Agents Chemother 1995; 39:661–7.
13. Dattwyler RJ, Luft BJ, Kunkel M, et al.. Ceftriaxone compared with doxycycline for the treatment of acute disseminated Lyme disease. N Engl J Med 1997; 337:289–94.
14. Fitzmaurice GM, Laird NM, Ware JH. Applied Longitudinal Analysis. Hoboken, N.J. Wiley-Interscience, ©2004; pp 391-4.
15. Klempner MS, Hu LT, Evans J, et al. Two controlled trials of antibiotic treatment in patients with persistent symptoms and a history of Lyme disease. N Engl J Med 2001;345(2):85–92.
16. Krupp LB, Hyman LG, Grimson R, et al. Study and treatment of post Lyme disease (STOP-LD): a randomized double masked clinical trial. Neurology 2003;60(12):1923–30.
17. Fallon BA, Keilp JG, Corbera KM, Petkova E, Britton CB, Dwyer E, Slavov I, Cheng J, Dobkin J, Nelson DR, Sackeim HA. A randomized, placebo-controlled trial of repeated IV antibiotic therapy for Lyme encephalopathy. Neurology 2008;70:992-1003.
18. Dattwyler RJ, Halperin JJ, Pass H, Luft BJ. Ceftriaxone as effective therapy for refractory Lyme disease. J Infect Dis 1987;155:1322–5.
19. Dattwyler RJ, Halperin JJ, Volkman DJ, Luft BJ. Treatment of late Lyme borreliosis—randomized comparison of ceftriaxone and penicillin. Lancet 1988; 1:1191–4.
20. Logigian EL, Kaplan RF, Steere AC. Chronic neurologic manifestations of Lyme disease. N Engl J Med 1990; 323:1438–44.
21. Logigian EL, Kaplan RF, Steere AC. Successful treatment of Lyme encephalopathy with intravenous ceftriaxone. J Infect Dis 1999;180:377–83.
Competing interests
None
Bottom Line: Patient Choice is Paramount
17 July 2010
The Infectious Diseases Society of America (IDSA) Lyme guidelines place commercial interests above quality of patient care, leaving seriously ill patients without effective treatment options. The narrow IDSA diagnostic criteria miss half the cases of Lyme disease, and up to half or more of the patients treated according to these guidelines remain ill. This is not acceptable.
The IDSA guidelines understate the seriousness of chronic Lyme disease, comparing ongoing Lyme symptoms to the "aches and pains of daily living." In contrast, peer-reviewed published studies show that people with chronic Lyme disease have disability and pain comparable to that of people with congestive heart failure and post-surgical pain, respectively. Schoolchildren with Lyme disease are classified as disabled and qualify for accommodations under Section 504 of the Americans with Disabilities Act. Deaths related to Lyme disease have also been reported.
The IDSA guidelines overstate the risks of long-term antibiotic treatment. Long-term treatment under the care of a qualified health care professional has been found to be safe and effective for a number of infectious diseases. The guidelines limit clinical judgment and hold physicians to arbitrary and unproven formulas. The IDSA-endorsed practice of terminating treatment despite persisting symptoms may result in advanced neurological injury, disability and death and constitutes medical negligence.
The California Lyme Disease Association (CALDA) recommends that patients with Lyme disease see a doctor affiliated with the International Lyme and Associated Diseases Society, who will individualize their treatment plan and consider patient preferences. Patients have the right to make informed choices based on their own personal values, just like patients with cancer or other diseases. In our opinion anything less is immoral, illegal, and inhumane.
Phyllis Mervine, EdM
President, CALDA
Editor-in-Chief, Lyme Times
Competing interests
None