Improving Diagnostic Accuracy

“Have you or your family or close friends ever had a missed or delayed diagnosis?” That’s a question Michael Kanter, MD, medical director of quality and clinical analysis for the Southern California Permanente Medical Group, often asks when he talks to groups of physicians. Invariably, approximately 30 percent of the audience members raise their hands in response. Most of those physicians probably assume they were unlucky outliers, but Kanter knows better.

“All doctors are concerned about being accurate in a diagnosis — that’s in a physician’s DNA,” he says. “But are they aware that 1 in 20 Americans in an ambulatory setting experiences a diagnostic error each year? I don’t think they realize the magnitude of the problem.”

Few people realized the magnitude until the National Academy of Medicine launched a comprehensive effort in 2015 to examine diagnostic errors. The academy’s predecessor, the Institute of Medicine, in 1999 released a report entitled To Err Is Human, which revealed extensive patient-safety problems in American healthcare, but the role of diagnostic errors did not get a sniff.

The NAM study committee did not make the same mistake. Based on its proposed definition of diagnostic errors, “The failure to establish an accurate and timely explanation of the patient’s health problem(s) or communicate that explanation to the patient,” it concluded that diagnostic errors contribute to about 10 percent of patient deaths. In fact, an estimated 12 million adult outpatients receive care that involves some kind of diagnostic problem each year.(1) That realization prompted creation of the Society to Improve Diagnosis in Medicine (SIDM), which brings together physician leaders who are committed to reducing the rate of diagnostic errors.

Among them is Dennis Torretti, MD, Geisinger Medical Center’s associate chief medical officer, who began attending the society’s meetings several years ago. “As I began to learn more, it became clear that this was an important undertaking,” he says. “And as physician leaders, it is part of our responsibility to begin addressing issues in this regard.”

Being Willing To Look

The first, and perhaps most important, leadership task in addressing misdiagnosis is to create a work environment in which clinicians are not afraid of the topic, says Robert Trowbridge, MD, a hospitalist and director of undergraduate medical education curriculum at Maine Medical Center. He is also co-director of the Introduction to Clinical Reasoning course at Tufts University School of Medicine and an SIDM member.

Delayed or missed diagnoses occur for two reasons: system failures, such as lab results that are not communicated properly, and cognitive errors. In many cases, a combination of the two is the reason.

“Framing it as a multi-dimensional problem that has multiple contributors and multiple possible fixes goes a long way toward clinicians being open about this because they recognize that it’s not just them making a mistake,” Trowbridge says.

At Maine Medical Center, the topic of diagnostic errors is “normalized” by including the category in a global error-reporting system. “When you sign into that system, diagnostic issues is one of the tiles that you see — right next to medication errors or falls or whatever type of error you might want to report,” he says.

Similar to other errors reported in the system, diagnosis-related cases are subject to root-cause analysis in search of various factors that might have contributed. Even if a clinician’s reasoning is part of the problem, analysis digs deeper to explore whether too many patients were included on that day’s schedule or whether support was insufficient for a complex case load.

Maine Medical Center’s diagnostic error workgroup applies the error reports to their initiatives to improve diagnostic safety across the organization. It does not delve into specific cases, but rather advocates for protocols and processes that would prevent future diagnostic errors.

“Several of us are pretty well-versed in diagnostic error, so we have a sense of what would be the highest impact things that we can do to improve diagnostic safety,” Trowbridge says.

The workgroup, co-chaired by the medical center’s vice president for patient safety and director of risk management, includes physicians, nurses, and laboratory and radiology staff. Trowbridge says the multidisciplinary membership increases the likelihood that good solutions will be found — and that those solutions will have broad buy-in. The workgroup championed the medical center’s recent adoption of open notes, which is the sharing of visit notes with patients (and, if they wish, with their family members or caregivers) so they can confirm that the notes are thorough and accurate. The group is also considering a “chain of information” for laboratory findings to make sure the appropriate person is alerted promptly in case of abnormal results so that timely action can be taken.

Tackling Errors At the System Level

Diagnostic errors can occur even when the physician is doing everything right.

“Test results come back to a doctor, they are abnormal, and then there’s no follow-up — every physician leader over quality sees this issue,” says Kanter, executive vice president and chief quality officer for The Permanente Federation.

More than a decade ago, Kanter and other Kaiser Permanente leaders began working on this issue for a specific diagnosis — chronic kidney disease — and ended up as diagnostic-improvement pioneers.

Their KP SureNet technology scans patients’ electronic health records for abnormal or missing test results and notifies clinicians if there has been no proper follow-up.

For example, if an elevated creatinine level shows up in a blood test and the patient does not have a diagnosis of kidney disease, the patient should be advised to have a second test three months later; if the creatinine level is still high, the physician needs to consider a diagnosis of chronic kidney disease. Over a four-year period, SureNet prompted physicians in 12,396 instances that a second test was needed, and 30 percent of those confirmed chronic kidney disease.

KP SureNet has grown to include about 30 tracking and alert programs that are improving the number of timely and accurate diagnoses.

In Kaiser’s Southern California region, the technology identified 9,381 men with an elevated or abnormal prostate-specific antigen (or PSA) screening test who had not been evaluated by a urologist within 12 weeks of receiving results. After SureNet flagged their clinicians about the needed follow-up, about 1,000 men were diagnosed with prostate cancer.

In Kaiser’s Colorado region, SureNet identified 3,139 patients who were at high risk of stroke but who were not on anti-coagulation therapy. Within 17 months of the alert, more than 40 percent were either prescribed medication or, after a shared decision-making conversation with their physician, had decided not to medicate.

“With its ability to intercept diagnostic errors before there is harm, the SureNet program represents a model that, in 10 years, every healthcare system in the country will be using,” Mark Graber, MD, SIDM’s founder and former president, said in a press release announcing SureNet’s results.

Improving Cognitive Reasoning

In academic medical centers, teaching trainees how to make the right diagnosis — and how to avoid common cognitive pitfalls — is standard work, says Robert M. Wachter, MD, chair of the Department of Medicine at the University of California-San Francisco; however, at community hospitals that lack training programs, even seasoned physicians may not be aware of the cognitive pitfalls.

“I think you could run into a bit of ‘Diagnosis is what I do and I’ve been doing it for 30 years — what do I need to learn?’ ” he says.

The answer might be a surprise. Until recently, most physicians were not systematically trained in metacognition, which is defined as being aware of your own thinking and which is needed to avoid diagnostic mistakes.

At UCSF, faculty and residents learn when cases are presented to Gurpreet Dhaliwal, MD, a professor of medicine who studies the cognitive processes used in diagnostic reasoning as a learning exercise. At morbidity and mortality conferences, leaders dissect cases that have an identified diagnostic error to demonstrate to participants not only obvious problems such as a missed symptom or an incorrect test, but also problems in cognitive reasoning. Those problems include anchoring bias (relying too heavily on an initial piece of information) and confirmation bias (the tendency to interpret new evidence as confirmation of an existing theory). “Our residents now have this skill, that we certainly weren’t taught, which is to think about diagnostic pitfalls in real time,” Wachter says.

Physician leaders must create an environment in which their colleagues will be open to discussing their errors as well as those of others if they know they will not be treated unfairly for being honest.

“It has to be very clear that the goal here is not finger-pointing, the goal is not to shame people,” he says. “We all can be better at this, and let’s try to learn how to get better at it.”

Learning From Mistakes

Every provider organization experiences diagnostic errors, and each can be a learning lab for improvement. Leaders at Geisinger Medical Center are developing a formal protocol for identifying diagnostic errors, discussing them with the individuals involved, and sharing the lessons learned so the whole organization benefits.

“We want to make every diagnostic error a learning opportunity,” Torretti says.

Geisinger’s interest in diagnostic improvement began when executive leaders decided that the medical center needed to get its arms around the complex topic. They established a diagnostic error exploration team, inviting members of the medical informatics group and clinical champions who were interested in the topic to join in.

“We simply began to discuss the topic,” Torretti says. “There was no clear-cut road map or pathway that’s been outlined. It’s a difficult topic but we felt it important that we needed to be engaged in doing something.”

The discussions led to creation of a workgroup charged with identifying areas of opportunity, defining program goals, and developing a working definition of diagnostic error. That group decided that if diagnostic improvement was going to get the attention it needed, a more formal structure was necessary.

To that end, Geisinger’s Committee to Improve Clinical Diagnosis, a subcommittee of the system’s Peer Review Committee, has a charter and a formal reporting structure.

“It has broad representation from all the major disciplines, as well as risk and patient safety and quality representation,” Torretti says. “And we report to the clinic leadership twice a year on the work that we’re doing.”

Over the past year, Geisinger has been working in partnership with Hardeep Singh, MD, MPH, director of the Diagnosis Improvement Safety Center based at the Houston VA Center for Innovations in Quality, Effectiveness and Safety and Baylor College of Medicine, to develop a learning lab for diagnosis. This lab, known as Safer Dx Learning Lab, aims to generate new knowledge, innovations, tools, and strategies related to reducing diagnostic error, many of which will be useful for other healthcare leaders.

Cases are identified from several sources, including risk and patient safety data. Following a preliminary review, the cases are sent to the Committee to Improve Clinical Diagnosis for further assessment.

“As we began reviewing these cases and categorizing them, we realized that we needed to address the issues identified,” Torretti says. “But it wasn’t necessarily the committee alone that was needed. It was necessary to engage individual departments and work units in the process.”

In the event of a diagnostic error, the multidisciplinary committee reviews the case and identifies issues and potential learning opportunities. The case summary and committee review are forwarded to the appropriate department director and that department’s quality director to begin a four-step process.

First, the department leaders discuss the case with the individual or individuals involved.

“This is meant to be an informal, constructive learning opportunity — it can’t be viewed as punitive or negative in any way, shape, or form,” Torretti says. “It is meant to be a two-way dialogue.”

Second, if there are lessons to be learned from the case, the department is expected to discuss the case and the learning opportunities at a staff meeting or clinical conference, allowing others to learn and benefit from the information. The clinical narrative can be an important learning tool.

Next, if processes need to be changed to prevent the same problem in the future, an action plan must be developed and implemented.

Finally, if system-level issues contributed to the problem, the department leaders are asked to notify Torretti’s team so it can address them.

Early Steps Of A Long Journey

Singh’s research team is working on the basic science of diagnostic error and improving diagnoses. According to Singh, recent research related to delays in cancer diagnosis found that physicians are fearful of both giving and receiving feedback about a delayed diagnosis.

His work with Geisinger seeks to advance understanding of what and what does not work to engage physicians in improving diagnoses with the goal of sharing that knowledge broadly. “How do we get physicians to reflect on their own diagnosis? I don’t think we know how to do that really well yet,” he says.

While that is being explored, physician leaders can do several helpful things in their own organizations. Most, however, have not yet started focusing attention and resources on diagnostic improvement.

“I would say few health systems are working on this seriously — very few,” Singh says.

Those that are, typically are working on a single issue, such as improving relay of test results to patients or making sure patients get the correct follow-up after a screening colonoscopy, for example. While those are important efforts, Singh encourages all provider organizations to start systematically collecting and analyzing data about their diagnostic errors.

“Measurement is the key thing that all health systems ought to be doing,” he says. “It’s data that they probably already have access to from risk-management reports or patient complaints or executive concerns or reports from providers.” Because errors don’t reveal themselves easily, there is no easy way to measure rates of diagnostic error. If a patient who goes to the emergency department complaining of dizziness gets sent home and shortly thereafter suffers a stroke that is treated in a different hospital, the emergency physician will never know about the missed diagnosis, and the second hospital where the patient was treated is not responsible for the mistake.

Thus, “measuring” diagnostic errors for health system leaders means beginning to look at their health systems to identify some cases, create a list, and develop a way to systematically ensure that cases are added when appropriate.

“Even if you only get to know about a few errors per month, getting into details of understanding why your health system is having these issues is really important,” Singh says. “This measurement for improvement matters.”

The diagnostic error problem requires leaders willing to spend time, energy, and social capital without expecting the quick rewards their efforts bring elsewhere.

“Even if you start working on it, you may not know if you’re making any progress, because there are no good measures and there is not any obvious business case,” Kanter says.

But that doesn’t mean it is not important.

“I put this on the moral imperative list because there’s an ethical obligation to provide the right diagnosis and to figure out ways to decrease the rate at which the right diagnosis is not made,” he says. “We are at a very early stage on a long journey.”

Reference

1. Singh H, Meyer A, Thomas EJ. The Frequency of Diagnostic Errors in Outpatient Care: Estimations from Three Large Observational Studies Involving US Adult Populations. BMJ Qual Saf. 2014;23:727–31.