Innumeracy at the Centers for Medicare & Medicaid Services (CMS)

<div class="MsoNormal">A physician participant in an American Medical Informatics Association listserv I subscribe to had some time ago commented that the CMS, in their requirement that physicians meet certain percentages on measures regarding adherence to the "Meaningful Use" incentive program for EHRs, were calculating those percentages incorrectly.</div><div class="MsoNormal"><br /></div><div class="MsoNormal">He followed up with an actual response from CMS, and has given me permission to repost.  I am redacting his name:</div><div class="MsoNormal"><br /></div><blockquote class="tr_bq"><div class="MsoNormal">... I have posted previously that the CMS calculator found at <a href="https://exchangeweb.drexel.edu/owa/redir.aspx?C=mnsZV_x_TUmIjS3hVz8N2LbSo8knNNEIzqgX1E2MFvWksVZnCP86blNjPEz3cFW5L-p0fS64-Xw.&URL=http%3a%2f%2fwww.cms.gov%2fRegulations-and-Guidance%2fLegislation%2fEHRIncentivePrograms%2fStage_2.html" target="_blank">http://www.cms.gov/Regulations-and-Guidance/Legislation/EHRIncentivePrograms/Stage_2.html</a> appears to calculate improperly.  Last night I received a response from CMS regarding my concern:</div><div class="MsoNormal"><br /></div><div class="MsoNormal" style="margin-left: .5in;">“Dear Mr. (sic) [name redacted],</div><div class="MsoNormal" style="margin-left: .5in;"><br /></div><div class="MsoNormal" style="margin-left: .5in;">Thank you for your recent inquiry. Below is the information you requested about the attestation calculator.</div><div class="MsoNormal" style="margin-left: .5in;"><br /></div><div class="MsoNormal" style="margin-left: .5in;">As adopted by CMS, measures generally require that providers achieve "more than" a specific percentage. <b>The language in these measures is clear; "more than 50 percent" would equate to 51 percent or more.</b> This has consistently been the policy from the beginning of the program.</div><div class="MsoNormal" style="margin-left: .5in;"><br /></div><div class="MsoNormal" style="margin-left: .5in;">The attestation calculator now rounds fractions of a percent to the closest whole number. This correctly identifies when the required performance level is reached.</div><div class="MsoNormal" style="margin-left: .5in;"><br /></div><div class="MsoNormal" style="margin-left: .5in;">The tracking number for this inquiry is 2014-XXXXX. . . . “</div><div class="MsoNormal"><br /></div><div class="MsoNormal"><br /></div><div class="MsoNormal">My examples included a numerator of 549 with a denominator of 10,000 for a metric that required “greater than 5%”.  These are screen shots of the submission:</div></blockquote><br /><br /><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhz4oQ46S84r4Sngb24v9Pa9ZGQwMsdKclLNIrq5OxygKMLJGGKhi8FHYQyCVhakiWWGKCj_wYEPDAVkgBA_La0WQxm0UiKpVx9XWgp61BWs_Ot8HivmbQJu11s0bx0yLQlxc0ty6HrHVbX/s1600/calc.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="335" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhz4oQ46S84r4Sngb24v9Pa9ZGQwMsdKclLNIrq5OxygKMLJGGKhi8FHYQyCVhakiWWGKCj_wYEPDAVkgBA_La0WQxm0UiKpVx9XWgp61BWs_Ot8HivmbQJu11s0bx0yLQlxc0ty6HrHVbX/s1600/calc.jpg" width="400" /></a></div><br /><blockquote class="tr_bq"><div class="MsoNormal"></div><div class="MsoNormal" style="margin-left: .5in;"></div><div class="MsoNormal"><br /></div><div class="MsoNormal">Note that the calculator appears to use 3 significant figures, but in fact rejects the calculation by rounding to only 1 significant figure.  </div><div class="MsoNormal"><br /></div><div class="MsoNormal">Years ago at a CMS conference I remember the speaker remarking that physicians and hospitals only had to enter the numbers “and we could agree on the arithmetic.”  I learned in fourth grade how to round numbers, and how to count significant digits.  I do not agree with their arithmetic, nor do I accept their determination that “more than 50 percent" would equate to 51 percent or more”.  Granted, one hopes that those attempting to attest will not have numbers that skim so closely to the metric limit, but it could happen.</div></blockquote><div class="MsoNormal"><br /></div><div class="MsoNormal">I note in the reply from CMS this statement:</div><blockquote class="tr_bq"><div class="MsoNormal"><br /></div><b>"The language in these measures is clear; 'more than 50 percent' would equate to 51 percent or more."</b></blockquote><br />I emphasize:  <span style="color: red;"><span style="font-size: large;"><b> </b></span></span><br /><br /><span style="color: red;"><span style="font-size: large;"><b>THE - LANGUAGE - IN - THESE - MEASURES - IS - CLEAR.</b></span></span><br /><br />Here are  actual instructions for, say, CPOE measurement on medication ordering (<a href="http://www.cms.gov/Regulations-and-Guidance/Legislation/EHRIncentivePrograms/downloads/Stage2_EPCore_1_CPOE_MedicationOrders.pdf">http://www.cms.gov/Regulations-and-Guidance/Legislation/EHRIncentivePrograms/downloads/Stage2_EPCore_1_CPOE_MedicationOrders.pdf</a>):<br /><br /><blockquote class="tr_bq"><u><b>Attestation Requirements</b></u><br /><u><b>DENOMINATOR/NUMERATOR/THRESHOLD/EXCLUSION</b></u><br /><br />Measure 1: Medication<br /><br />DENOMINATOR: Number of medication orders created by the EP during the EHR reporting period.<br /><br />NUMERATOR: The number of orders in the denominator recorded using CPOE.<br /><br />THRESHOLD: The resulting percentage must be more than 60 percent in order for an EP to meet this measure.<br /><br />EXCLUSION: Any EP who writes fewer than 100 medication orders during the HER (sic) reporting period. </blockquote><br />Now, I had a perfect "800" in math on my SAT test in 1974, so (while probably excluding me from some employment in today's culture of stupid due to having "too much experience"), I do believe I am perfectly well qualified to determine that "more than 60 percent" would mean, anything more than 60.0, such as, say, 60.1%.<br /><br />Let's consider possible explanations.<br /><br /><u><b>Possibility (1)</b></u><br />Perhaps CMS is performing their calculations with an old Pentium-1 computer whose CPU has the FDIV bug causing math errors (<a href="http://en.wikipedia.org/wiki/Pentium_FDIV_bug">http://en.wikipedia.org/wiki/Pentium_FDIV_bug</a>)?<br /><br />On second thought:<br /><br />I don't think that is a good explanation. <br /><br /><u><b>Possibility (2)</b></u><br />Perhaps I have become demented from too much exposure to the numeracy-challenged since 1974?<br /><br />An experiment is in order:<br /><br />In 1969, we landed men on the Moon.  It took a lot of math to perform that feat, and miscalculation would have led to disaster, for example on the 25,000-MPH re-entry into Earth's atmosphere at precisely the correct angle required to avoid burn up, or skipping off the atmosphere like a stone off a pond, back out into space to be lost forever.<br /><br />I learned computing about that time on a Digital Equipment Corporation PDP-8/S, a computer from that era (<a href="http://cci.drexel.edu/faculty/ssilverstein/medinformatic1/gw-pdp8s.htm">http://cci.drexel.edu/faculty/ssilverstein/medinformatic1/gw-pdp8s.htm</a>).<br /><br />I wrote and ran a small program on a PDP-8 emulator WinEight (available free at <a href="http://www.sparetimegizmos.com/Software/WinEight.htm">http://www.sparetimegizmos.com/Software/WinEight.htm</a>) that perfectly emulates the old PDP-8.  Here is the program in the programming language FOCAL, 1969 version:<br /><br /><blockquote class="tr_bq">CONGRATULATIONS!!<br />YOU HAVE SUCCESSFULLY LOADED 'FOCAL,1969' ON A PDP-8 COMPUTER.<br /><br />SHALL I RETAIN LOG, EXP, ATN ?:YES<br /><br />PROCEED.<br /><br />*1.1 SET P=(601/1000)*100<br />*1.2 IF (P-60) 1.3, 1.4, 1.5<br />*1.3 TYPE "P IS LESS THAN 60%";G 1.6<br />*1.4 TYPE "P EQUALS 60%";G 1.6<br />*1.5 TYPE "P IS MORE THAN 60%"<br />*1.6 TYPE !;QUIT</blockquote><br />The program sets variable "P" to 601/1000, then multiplies by 100 to derive a percent.  It then compares the result stored in "P" to 60, and branches to the correct statement regarding the relation of P to 60%.  Let's run the program:<br /><br /><blockquote class="tr_bq">*<br />*GO<br /><b>P IS MORE THAN 60%</b><br />*</blockquote><br />Good news. <br /><br />I guess I am not demented after all.<br /><br /><u><b>Possibility (3)</b></u><br />Someone (or a group of people) at CMS doesn't or don't realize that in large datasets, <b>percentages are often not integer whole numbers.</b><br /><br />Possibility #3 can be illustrated best by a cartoon:<br /><br /><br /><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody><tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiCRwf49Ok93IT5g9puPUujkyj3ul4BmKe8A8RqrIo0xa-IaaqSuPQaQrL1HN0B62MUgOK7Lcam4QlpMYcJK7erGTMTlgEUngVZ2fkD5omr5yNOIa8vwH_dDUQNEFcdUckR1lsJtHNkEyXH/s1600/dunce.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="200" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiCRwf49Ok93IT5g9puPUujkyj3ul4BmKe8A8RqrIo0xa-IaaqSuPQaQrL1HN0B62MUgOK7Lcam4QlpMYcJK7erGTMTlgEUngVZ2fkD5omr5yNOIa8vwH_dDUQNEFcdUckR1lsJtHNkEyXH/s1600/dunce.jpg" width="133" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;">Click to enlarge.  1+3 = 13, right?</td></tr></tbody></table><br /><br />Or perhaps this:<br /><br /><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody><tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiUDXpKjjMtOtSTY_6Jibyh2Cf2AQJvS5BhjbKbKf43JslhQ3Ly9hKSQNypPSiY2v7EsycBsKEoCNGL3uf8ankUN8BXdRawaP6xZDCyqbNCramZgQkeYYs72OwJQ806XeQiW_JZkQBofhv8/s1600/Butterfinger-math.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="152" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiUDXpKjjMtOtSTY_6Jibyh2Cf2AQJvS5BhjbKbKf43JslhQ3Ly9hKSQNypPSiY2v7EsycBsKEoCNGL3uf8ankUN8BXdRawaP6xZDCyqbNCramZgQkeYYs72OwJQ806XeQiW_JZkQBofhv8/s1600/Butterfinger-math.jpg" width="200" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;">Math for Underachievers.  Click to enlarge.</td></tr></tbody></table><br />I leave it to the reader to determine which of the three possibilities is the explanation for the assertion by CMS that <b>"The language in these measures is clear; 'more than 50 percent' would equate to 51 percent or more."</b><br /><br />Note: this post was meant as satire and humor and irony, but innumeracy in our healthcare regulators is anything but funny.<b> </b><br /><br />-- SS

Innumeracy at the Centers for Medicare & Medicaid Services (CMS)

A physician participant in an American Medical Informatics Association listserv I subscribe to had some time ago commented that the CMS, in ...

Đọc thêm »