Innumeracy at the Centers for Medicare & Medicaid Services (CMS)

A physician participant in an American Medical Informatics Association listserv I subscribe to had some time ago commented that the CMS, in their requirement that physicians meet certain percentages on measures regarding adherence to the "Meaningful Use" incentive program for EHRs, were calculating those percentages incorrectly.

He followed up with an actual response from CMS, and has given me permission to repost.  I am redacting his name:

... I have posted previously that the CMS calculator found at http://www.cms.gov/Regulations-and-Guidance/Legislation/EHRIncentivePrograms/Stage_2.html appears to calculate improperly.  Last night I received a response from CMS regarding my concern:

“Dear Mr. (sic) [name redacted],

Thank you for your recent inquiry. Below is the information you requested about the attestation calculator.

As adopted by CMS, measures generally require that providers achieve "more than" a specific percentage. The language in these measures is clear; "more than 50 percent" would equate to 51 percent or more. This has consistently been the policy from the beginning of the program.

The attestation calculator now rounds fractions of a percent to the closest whole number. This correctly identifies when the required performance level is reached.

The tracking number for this inquiry is 2014-XXXXX. . . . “

My examples included a numerator of 549 with a denominator of 10,000 for a metric that required “greater than 5%”.  These are screen shots of the submission:

Note that the calculator appears to use 3 significant figures, but in fact rejects the calculation by rounding to only 1 significant figure.  

Years ago at a CMS conference I remember the speaker remarking that physicians and hospitals only had to enter the numbers “and we could agree on the arithmetic.”  I learned in fourth grade how to round numbers, and how to count significant digits.  I do not agree with their arithmetic, nor do I accept their determination that “more than 50 percent" would equate to 51 percent or more”.  Granted, one hopes that those attempting to attest will not have numbers that skim so closely to the metric limit, but it could happen.

I note in the reply from CMS this statement:

"The language in these measures is clear; 'more than 50 percent' would equate to 51 percent or more."

I emphasize:   

THE - LANGUAGE - IN - THESE - MEASURES - IS - CLEAR.

Here are  actual instructions for, say, CPOE measurement on medication ordering (http://www.cms.gov/Regulations-and-Guidance/Legislation/EHRIncentivePrograms/downloads/Stage2_EPCore_1_CPOE_MedicationOrders.pdf):

Attestation Requirements
DENOMINATOR/NUMERATOR/THRESHOLD/EXCLUSION

Measure 1: Medication

DENOMINATOR: Number of medication orders created by the EP during the EHR reporting period.

NUMERATOR: The number of orders in the denominator recorded using CPOE.

THRESHOLD: The resulting percentage must be more than 60 percent in order for an EP to meet this measure.

EXCLUSION: Any EP who writes fewer than 100 medication orders during the HER (sic) reporting period.

Now, I had a perfect "800" in math on my SAT test in 1974, so (while probably excluding me from some employment in today's culture of stupid due to having "too much experience"), I do believe I am perfectly well qualified to determine that "more than 60 percent" would mean, anything more than 60.0, such as, say, 60.1%.

Let's consider possible explanations.

Possibility (1)
Perhaps CMS is performing their calculations with an old Pentium-1 computer whose CPU has the FDIV bug causing math errors (http://en.wikipedia.org/wiki/Pentium_FDIV_bug)?

On second thought:

I don't think that is a good explanation.

Possibility (2)
Perhaps I have become demented from too much exposure to the numeracy-challenged since 1974?

An experiment is in order:

In 1969, we landed men on the Moon.  It took a lot of math to perform that feat, and miscalculation would have led to disaster, for example on the 25,000-MPH re-entry into Earth's atmosphere at precisely the correct angle required to avoid burn up, or skipping off the atmosphere like a stone off a pond, back out into space to be lost forever.

I learned computing about that time on a Digital Equipment Corporation PDP-8/S, a computer from that era (http://cci.drexel.edu/faculty/ssilverstein/medinformatic1/gw-pdp8s.htm).

I wrote and ran a small program on a PDP-8 emulator WinEight (available free at http://www.sparetimegizmos.com/Software/WinEight.htm) that perfectly emulates the old PDP-8.  Here is the program in the programming language FOCAL, 1969 version:

CONGRATULATIONS!!
YOU HAVE SUCCESSFULLY LOADED 'FOCAL,1969' ON A PDP-8 COMPUTER.

SHALL I RETAIN LOG, EXP, ATN ?:YES

PROCEED.

*1.1 SET P=(601/1000)*100
*1.2 IF (P-60) 1.3, 1.4, 1.5
*1.3 TYPE "P IS LESS THAN 60%";G 1.6
*1.4 TYPE "P EQUALS 60%";G 1.6
*1.5 TYPE "P IS MORE THAN 60%"
*1.6 TYPE !;QUIT

The program sets variable "P" to 601/1000, then multiplies by 100 to derive a percent.  It then compares the result stored in "P" to 60, and branches to the correct statement regarding the relation of P to 60%.  Let's run the program:

*
*GO
P IS MORE THAN 60%
*

Good news.

I guess I am not demented after all.

Possibility (3)
Someone (or a group of people) at CMS doesn't or don't realize that in large datasets, percentages are often not integer whole numbers.

Possibility #3 can be illustrated best by a cartoon:

Click to enlarge.  1+3 = 13, right?

Or perhaps this:

Math for Underachievers.  Click to enlarge.

I leave it to the reader to determine which of the three possibilities is the explanation for the assertion by CMS that "The language in these measures is clear; 'more than 50 percent' would equate to 51 percent or more."

Note: this post was meant as satire and humor and irony, but innumeracy in our healthcare regulators is anything but funny.

-- SS