healthy4u: SICC syndrome

Who was at fault here? UK Woman left to die after computer decision support blunder

Who was at fault here? Those who modified the fall height parameters, those who designed the decision support system such that it could override life threatening problems based on a single parameter, endusers, their managers, or all? Or was the problem the syndrome of inappropriate confidence in computers (SICC syndrome)? I opine all of the above, in this cautionary tale: <a href="http://www.telegraph.co.uk/health/healthnews/7489669/Woman-left-to-die-after-999-ambulance-blunder.html"></a><blockquote><a href="http://www.telegraph.co.uk/health/healthnews/7489669/Woman-left-to-die-after-999-ambulance-blunder.html">Woman left to die after 999 ambulance blunder</a> By Laura Donnelly, Health Correspondent Telegraph.co.uk Published: 9:00AM GMT 21 Mar 2010 An investigation into a woman’s death has exposed a catastrophic decision by ambulance chiefs which may have cost hundreds of lives. The blunder arose when call centre staff were not warned of flaws with a computer system that prioritises emergencies before dispatching ambulances. Bonnie Mason, 58, fell down the stairs and died from a head injury after 999 controllers in Suffolk failed to identify her situation as “life-threatening”. An investigation by The Sunday Telegraph has uncovered a critical danger placed in the software used by most ambulance services. For years, 999 calls in life-threatening situations like Mrs Masons’s were accidentally “downgraded”, with call handlers told not to send the most urgent response. While some services spotted the risk, ordering operatives to override the computer’s orders manually, five of England’s 12 ambulance trusts did not allow call handlers to upgrade such calls [A belief that "the computer is omniscient" seems to be the only basis for such an exclusion of human judgment - ed.] They include the East of England ambulance service, which covers Suffolk and which only identified the risk after Mrs Mason’s death. The danger in the system was created by the country’s most senior ambulance officials as they altered the program used by most control centres in an attempt to manage demand for 999 services. Most ambulance services use an international computerised system designed in America. In the US version, a fall of more than 6ft receives the maximum priority response. However, the government committee which governs its use in this country decided that such cases should be deemed less urgent [what were they thinking? - ed], and excluded from an eight minute category A target response time. In doing so, they created a potentially lethal flaw in the system. It meant that if a call involved a fall of more than 6ft it was designated a lower priority – a category B response – despite the presence of life-threatening conditions which were supposed to receive the most urgent category A response. [NOTE: If the other life threatening conditions were ignored by the computer system after its "first-pass" look at the height of the fall, then who actually created the most severe flaw is unclear to me, those who altered the parameter or those who designed the overall decision support logic - ed.] As a result, Mrs Mason lay unconscious for more than 38 minutes. The first ambulance sent to her home in the village of Eye, Suffolk, was diverted to attend to a drunk woman who had fallen on a pavement 22 miles away in Thetford, Norfolk. Because the inebriated woman had fallen at ground level, her situation was prioritised over that of Mrs Mason [perhaps because their was no entry of a "height of fall" - ed.], who was close to death by the time paramedics arrived. The East of England ambulance service, which also covers Bedfordshire, Cambridgeshire, Essex, Hertfordshire and Norfolk, said its operatives were instructed never to “override” the advice of the automated system.</blockquote> Ambulance dispatchers instructed to "never override the advice of the automated system?" Simply stunning if true. Read the whole story at the link above. On another note: government committees have rarely worked well in domains where critical thinking is essential. (I can't wait for the comparative effectiveness committees using flawed data from flawed EMR's to start their work. I'd written about that issue <a href="http://www.thefreelibrary.com/The+syndrome+of+inappropriate+overconfidence+in+computing:+an...-a0201551413">here</a>.) Finally, "Our policy is to always trust the computer" is not a way to run life-critical healthcare services. Ever. -- SS

Who was at fault here? UK Woman left to die after computer decision support blunder

Who was at fault here? Those who modified the fall height parameters, those who designed the decision support system such that it could ove...

Đọc thêm »

The Syndrome of Useless Information

<div style="text-align: justify;">I occasionally elevate comments and replies to the level of full posts if I feel they better illustrate and clarify significant points I raise. In my Jan. 9, 2010 post "<a href="http://hcrenewal.blogspot.com/2010/01/does-ceo-of-google-use-google.html">Does the CEO of Google Use Google? - And: Platform, Platform, Who's Got The Platform?</a>" I lamented the intrusion of ill-informed, reductionistic, "database platform"-centric views of non-clinical IT personnel into healthcare. I received the following feedback from an anonymous commenter: <blockquote>Anonymous wrote: Or then again, there's this: "Non-physicians may reach correct diagnoses by using Google: a pilot study." <a href="http://www.ncbi.nlm.nih.gov/pubmed/19130327">http://www.ncbi.nlm.nih.gov/pubmed/19130327</a> "Researchers found that almost six-in-10 difficult cases can be solved by using the world wide web as a diagnostic aid." <a href="http://www.dailymail.co.uk/news/article-415562/Doctors-using-Google-diagnose-illnesses.html">http://www.dailymail.co.uk/news/article-415562/Doctors-using-Google-diagnose-illnesses.html</a> I think this post [i.e., my HC Renewal post about Google's CEO and "platforms" - ed.] misses the larger point that Gawande and Schmidt were addressing (and to some extent agree on) - the "art" of the practice of medicine must be transformed to the science of the practice of medicine. And today, science and IT are necessarily intertwined in practical terms. The future will no doubt involve ever more sophisticated and useful clinical DSS. Historically speaking, invoking the complexity argument is fraught with issue as technological advances provide better and better ways to represent and manage it. January 16, 2010 10:03:00 AM EST</blockquote> Anonymous apparently meant these article references to exemplify the coming Age of Cybernetic Miracles in medicine. My response to this anonymous feedback was placed in the <a href="http://hcrenewal.blogspot.com/2010/01/does-ceo-of-google-use-google.html#c3758917515516282280">comment thread</a> of the "Does the CEO use Google" post and covered a number of bases. I've reproduced my response, and added additional explanatory notes not in my original response in bold red italics below: <blockquote><a href="http://www.blogger.com/profile/03994321680366572701" rel="nofollow" onclick="window.open(this.href);return false;">MedInformaticsMD</a> wrote: Re: Anonymous Jan 16 @ 10:03 AM- The article [the first, at nlm- ed.] you cite reports the following: Non-physicians may reach correct diagnoses by using Google: a pilot study. OBJECTIVE: We endeavoured to determine whether individuals who are not physicians are likely to arrive at correct diagnoses [note the stated objective carefully -- "are likely to arrive at correct diagnoses" - ed.] by using Internet resources. METHODS: In this prospective study four non-physicians used Google to search for diagnoses. They reviewed the 26 diagnostic cases presented in the case records of the New England Journal of Medicine during 2005; they were blind to the correct diagnoses. The main measurement was the percentage of correct diagnoses arrived at by non-physicians by using Google. The diagnostic success of the four non-physicians was compared to that of four young physicians. RESULTS: The average diagnostic success of non-physicians was 22.1% (95% confidence interval [CI] 4.5-39.7%). There was no statistically significant difference between the non-physicians regarding this outcome (p = 0.11). They took 8.9 +/- 6.7 (mean +/- standard deviation) minutes for case record reading and 17.4 +/- 7.9 minutes for Google searching per case. Non-physicians performed worse than physicians (50.9% [95% CI 37.4-64.5%]) in regard to diagnostic success (p <0.001). CONCLUSION: Non-physicians, at least those who have similar characteristics to the participants in the present study, may occasionally reach correct diagnoses by performing a brief web-based search. Doctors should realise that patients may assume a more active role in their health decision-making process and take this development into consideration in physician-patient interaction. -------------------------- This article suffers from what can be termed "the syndrome of useless information." [It is, in fact, a failure or negative finding with regard to its stated objective - ed.] Let's see: The average diagnostic success of non-physicians was 22.1% They omit the opposite semantic: that the google-armed nonphysicians got more than three quarters of the diagnoses WRONG. Will you trust your grandmother to them? Non-physicians performed worse than [young, a.k.a. inexperienced] physicians (50.9% [95% CI 37.4-64.5%]) in regard to diagnostic success (p <0.001). Young, inexperienced physicians (trainees or residents?) arriving at 51% of the diagnoses correctly using ONLY google (not, for example, Harrison's, and other standard medical texts that would not be highly usable to most non-physicians lacking background to fully understand it) is not unexpected. What is surprising is that the article omitted an essential control group: *experienced* physicians. It's not that they're hard to find. The article concludes: Non-physicians, at least those who have similar characteristics to the participants in the present study, may occasionally reach correct diagnoses by performing a brief web-based search. Occasionally reach correct diagnoses? How about "most often reach incorrect diagnoses?" Cf. a broken clock is still occasionally correct. [I repeat, the article is, in fact, a failure or negative finding with regard to its stated objective of "determining whether individuals who are not physicians are likely to arrive at correct diagnoses." They are in fact unlikely to get it right, with only a 22% hit rate, although the authors appear to have de-emphasized that fact. This will only make physicians' work harder as they "take this development into consideration", i.e., deal with patients armed with search engine-gleaned misdiagnoses - ed.] I do not view this article as revealing anything of practical value other than perhaps the dangers of allowing non-physicians armed with search engines to think they can perform medical diagnosis to any meaningful extent. Anonymous wrote: the "art" of the practice of medicine must be transformed to the science of the practice of medicine. And today, science and IT are necessarily intertwined in practical terms. Reductionist views of non-medical IT personnel spouting off about 'database platforms' will not advance the science or the art of medicine. Also, yes, IT is a tool of science; however, it and its designers and implementers facilitate science; the enablers of science are: scientists, using their insight, creativity, ingenuity, experience and expertise. Concerning the second article [at dailymail.co.uk - ed.], I note: "But they [the authors] stress the efficiency of the search and the usefulness of the retrieved information depend on the searchers' knowledge base." [The 'searchers' in this case were apparently - experienced physicians - ed.] In other words, search engines can facilitate experts. That is not exactly new knowledge. The 'art' of medicine, i.e., judgment, likewise is both indispensible, and irreproducible via "database platforms." Perhaps one day with advances in <a href="http://news.bbc.co.uk/2/hi/sci/tech/7740484.stm" rel="nofollow">cognitive computing</a> we will get to that point, but at present we can't even do as well as a cat. Note the statement from the IBM P.I. that "there are no computers that can even remotely approach the remarkable feats the mind perform." Until we get there, I think it's not unreasonable to hold off on non-clinicians touting reductionist information retrieval-centric views. Anonymous wrote: Historically speaking, invoking the complexity argument is fraught with issue as technological advances provide better and better ways to represent and manage it. You must not have read my original post. It is advances in information science (i.e., in informatics) that will provide those advances, not advances in [information] technology. IT is a tool; information science is an activity of the mind. Your statements clearly demonstrate a conflation of information technology and information science. Computers facilitate information science research, but they are certainly not its sine qua non.</blockquote> I should also add that the second article referenced by "Anonymous" refers to "obscure conditions such as Cushing's syndrome." I'm not exactly sure to whom Cushing's disease is an obscure disease. A non-medical IT person, perhaps? Now, back to reality. Computers serve as aids to clinicians, when the IT is "done well." However, there is no substitute -- except in sci fi -- for expertise. Medical expertise can only come from ~4 years of premed and then 4+ years of <a href="http://www.ischool.drexel.edu/faculty/ssilverstein/transcript_busm.JPG">hard medical study in a wide variety of preclinical and clinical sciences</a> leading to the M.D. degree, 3-4 years of postgraduate residency, and often 2+ additional years of <a href="http://www.ischool.drexel.edu/faculty/ssilverstein/transcript_yale.jpg">postdoctoral fellowship training</a> beyond residency for specialists (it should be noted, unfortunately, that there are no additional medical degrees beyond M.D., although residency and fellowship training often makes pursuing a Ph.D. in most fields seem like a cakewalk). Add to all that years of additional clinical experience in actual practice. <div style="text-align: justify;">I am quite disturbed that this study and experience has become so casually regarded outside of medicine, for example in the IT sector. I think it reflects poorly on the IT culture specifically, and on our current culture as a whole. </div></div><div style="text-align: justify;">Medical judgment is borne of that study and experience, not of healthcare computer dabbling typical of the business IT/MIS world in hospitals, and the all-too-often technical bachelor's degrees held by leaders in that domain. Google CEO Schmidt at least is a true computer scientist with a doctorate in CS; that does not, however, qualify him to comment on medical and medical informatics-related issues as in my <a href="http://hcrenewal.blogspot.com/2010/01/does-ceo-of-google-use-google.html">prior post</a>. Stated frankly: he and many others with similar views are venturing far outside their competencies and as a result are talking nonsensically. Incidentally, I believe it's time to move on from continually responding to the non-medical, IT-personnel proffered platformorrhea on how IT databases, decision support, artificial intelligence etc. will transform medicine from an art to a science, revolutionize medicine, and similar positions trafficked for at least several decades now. They are largely manifestations of the Syndrome of Inappropriate Overconfidence in Computing (SICC syndrome), a term I coined in the 1990's and described is some detail <a href="http://hcrenewal.blogspot.com/2008/12/21st-century-plague-syndrome-of.html">here</a> in a post entitled "A 21st Century Plague? The Syndrome of Inappropriate Over-Confidence in Computing." At that post I stated that: <blockquote>I, for one, would welcome a cessation of claims that IT will "revolutionize" any field that depends primarily on cognition, such as biomedicine, and a return to more temperate attitudes instead of the almost bellicose grandiosity about HIT we see today. That is to say, that HIT - with proper contributions from the aforementioned specialties [e.g., social science, social informatics, biomedical informatics, HCI, etc.] will facilitate better health care, not "revolutionize" it.</blockquote> In the future, similar comments as those from "Anonymous" will simply get referred back to the post you are now reading and the bolded one on SICC above. </div> Finally, I will not hold my breath for <a href="http://en.wikipedia.org/wiki/Emergency_Medical_Hologram" title="Emergency Medical Hologram" class="mw-redirect">Emergency Medical Hologram</a> Mark I, or R2D2 and C3PO to appear anytime soon. -- SS

The Syndrome of Useless Information

I occasionally elevate comments and replies to the level of full posts if I feel they better illustrate and clarify significant points I rai...

Đọc thêm »

Have we suffered a complete breakdown in the scientific method with regard to EHR and clinical IT?

Have we suffered a complete breakdown in the scientific method with regard to EHR and other clinical IT? I read announcements like this with trepidation: <div style="margin-left: 40px;"><a style="font-family: arial;" target="blank" href="http://govhealthit.com/articles/2009/03/31/sebelius-confirmation.aspx">http://govhealthit.com/articles/2009/03/31/sebelius-confirmation.aspx </a> “The goal,” Sebelius said, “is to provide every American with a safe, secure electronic health record by 2014." The nominee also endorsed efforts to use data gleaned from electronic medical records to conduct “comparative effectiveness research" (CER) to provide information on the relative strengths and weaknesses of alternative medical interventions to health providers and consumers.” </div> Recovery Act funds have been allocated to NIH specifically for comparative effectiveness research. NIH has further specified the definition of CER as: <blockquote>"[A] rigorous evaluation of the impact of different options that are available for treating a given medical condition for a particular set of patients. Such a study may compare similar treatments, such as competing drugs, or it may analyze very different approaches, such as surgery and drug therapy."</blockquote> NIH states that such research may include "the development and use of clinical registries, clinical data networks, and other forms of electronic health data that can be used to generate or obtain outcomes data as they apply to CER." The problems I foresee concern the word "rigorous" as in the above definition. The use of EHR data to reliably detect uncommon (but strong, discrete) early warning signals from a single drug or treatment -- to then be subject to more rigorous study with reasonable controls -- is itself a Medical Informatics "Grand Challenge." An example would be finding VIOXX's association with myocardial infarction earlier than we did, via an EHR-based automated postmarket surveillance process. Doing this is a "grand challenge" due to the nature of EHR data, which is as far from "clinical trials clean" as possible. It is what might be called highly uncontrolled. The statistical methods needed to reliably pull signals out of the muck for even a single drug are still exploratory, the problems formidable if one wants to stay scientifically sound. I wrote about the experimental nature of such efforts a few years ago <a target="blank" href="http://www.scribd.com/doc/12392184/A-Medical-Informatics-Grand-Challenge-the-EMR-and-PostMarketing-Drug-Surveillance-">here</a>, and believe an effort got underway at U. Indiana/Regenstrief to test such methodologies for postmarket surveillance about the same time. Now we have had what appears to be a leap of faith and logic of irrationally exuberant proportions, and probably a deviation from sound science as well. The government has announced enthusiasm for EHR data-based comparative effectiveness research (CER) not to aid science, but to cut costs (implying skipping the rigorous confirmatory phases) through elimination of more costly drugs and treatments deemed less effective or at effectiveness parity compared to less expensive choices. Following this thinking, perhaps in the future a metric will be developed for an "acceptable" improved benefit/cost ratio for expensive drugs that are better than cheaper alternatives? This overconfidence in EHR data is of concern. To detect relatively less concrete (i.e., than major ADE) "outcomes differences" between two or more drugs or treatments via EHR data - did treatment A lower blood pressure more than drug B, did drug C lessen depression more than drug D - rises to the level of "grand overconfidence in computing." To accomplish this task with reasonable scientific certainty from reams of EHR data, originating from different vendor systems, input by myriad people of different backgrounds with differing interpretations of terminologies (students/MD's/RN's etc) under different pressures (time, reimbursement maximization), and so forth, seems a stretch. What will the p values and predictive values be for such studies? Yet our incoming HHS secretary touts such methods? Ironically, the gold standard in medical science is the controlled clinical trial, yet EHR-based comparative effectiveness research itself as a research methodology, now touted by our government, seems to have gotten a pass. Even what I would consider minimum requirements for scientific treatment comparisons, such as well designed and reasonably controlled registries as developed <a href="http://www.ischool.drexel.edu/faculty/ssilverstein/failurecases/?loc=cases&sloc=Cardiology%20story">here for interventional cardiology</a>, with hundreds of granular, finely defined and "tuned" data elements, appear to be bypassed in EHR "miracle claims." Such precise registries take months or years to develop, implement, and train users to interact with properly. Further, such registries are not portable and must be created for individual medical domains and subdomains. Uncontrolled EHR data is no substitute for such efforts. The following question arises: Where are the comparative effectiveness studies that compare 1) EHR-based comparative effectiveness studies of drugs and treatments to 2) controlled clinical trials-based comparative effectiveness studies? In other words, where are the meta-clinical trials that compare EHR data mining-based comparative effectiveness research as a methodology, vs. the "traditional" gold standard methodology of controlled clinical trials to compare drugs or treatments? How do we know EHR-based CER studies will not produce GIGO that will cause harm through ham-fisted elimination or defunding of useful treatment options? While there are initial efforts underway to increase understanding of CER, e.g., <a href="http://grants.nih.gov/grants/funding/challenge%5Faward/High_Priority_Topics.pdf#topic_05">"Broad Challenge Area 5</a>" (PDF) of the NIH RC1 Challenge Grants in Health and Science Research, ominously, there is a lot of potential advantage to be had with terabytes of uncontrolled data and a political agenda. I fear that what will come from "comparative effectiveness research" that draws upon uncontrolled EHR data will be politics masquerading as comparative effectiveness research. Good luck to private practitioners and medical innovators. Good luck, pharma. Good luck, patients. This movement towards EHR uncontrolled data alchemy represents a further deviation from medical science towards the Syndrome of Inappropriate Over-Confidence in Computing (a.k.a. SICC Syndrome) writ large. It seems the IT industry has now rendered a scientific approach to HIT and its use obsolete. We see this "post scientific era" phenomenon in the takeover of clinical IT by vendors who contractually demand suppression of sharing of problems, we see it in a remarkably uncritical push for EMR's by 2014 now involving force of government (only financial at present, but will punitive licensure issues and other measures be off the table?) despite a growing body of literature advising caution, we see a consortium of big business/payers/vendors/myriad secondary feeder organizations gunning full blast for this technology without consideration of the possible downsides. Biomedical informatics, a scientific discipline (at least those parts of it not yet compromised by conflicts of interest), as a relevant field is very much a minority player in today's health IT. Even the contributions from experts and pioneers in the field of Biomedical Informatics, in the form of the Jan. 2009 National Research Council's report that "Current Approaches to U.S. Health Care Information Technology are Insufficient" (<a href="http://www.ischool.drexel.edu/faculty/ssilverstein/failurecases/?loc=cases&sloc=NRC">here</a>) has not had much impact. I see Biomedical Informatics' death as a relevant discipline that anyone of importance pays attention to, not too far down the road as well. -- SS Addendum April 20: We've seen this phenomenon in our economy. WSJ "Information Age" writer L. Gordon Crovitz <a href="http://online.wsj.com/article/SB124018430498933171.html">notes</a>: <blockquote>... In a paper for the scientific journal of the Royal Society back in 1994, Harvard economist Robert Merton wrote that "any virtue can become a vice if taken to extreme, and just so with the applications of mathematical models in finance practice." We know even better now that some risks can be calculated and thus reduced, while some unknowns cannot be turned into probabilities. "The mathematics of the models are precise, but the models are not, being only approximations to the complex, real world."</blockquote> I believe EMR data at best is a very loose approximation to the real world. It contains many "unknowns" regarding quality and reliability that cannot be turned into probabilities no matter how fancy the math. Asking too much of EHR data becomes a vice, not a virtue. -- SS

Have we suffered a complete breakdown in the scientific method with regard to EHR and clinical IT?

Have we suffered a complete breakdown in the scientific method with regard to EHR and other clinical IT? I read announcements like this with...

Đọc thêm »