Dr. Mostashari’s critique of our paper, will, we hope, open a fruitful  dialogue.  We trust that in the interest of fairness he will direct readers to  our response on his agency’s site. 
 [If not, HC Renewal posts get highly ranked by Google - ed.]Our study analyzed government survey data on a nationally representative  sample of 28,741 patient visits to 1187 office-based physicians.  We found that  electronic access to computerized imaging results (either the report or the  actual image) was associated with a 40% -70% increase in imaging tests,  including sharp increases in expensive tests like MRIs and CT scans; the  findings for blood tests were similar.  Although the survey did not collect data  on payments for the tests, it’s hard to imagine how a 40% to 70% increase in  testing could fail to increase imaging costs.
Dr. Mostashari’s statement that “reducing test orders is not the way that  health IT is meant to reduce costs” is surprising, and 
contradicts statements by his predecessor as National Coordinator that  electronic access to a previous CT scan helped him to avoid ordering a duplicate  and “saved a whole bunch of money.” 
A Rand study, widely cited by health IT advocates including  President Obama, estimated that health IT would save $6.6 billion annually on  outpatient imaging and lab testing.  
Another frequently quoted estimate of HIT-based savings  projected annual cost reductions of $8.3 billion on imaging and $8.1 billion on  lab testing.
We focused on electronic access to results because the common understanding  of how health IT might decrease test ordering is that it would facilitate  retrieval of previous results, avoiding duplicate tests.  Indeed, it’s clear  from the extensive press coverage that our study was seen as contravening this  “conventional wisdom”.
Nonetheless, Dr. Mostashari criticizes us for analyzing the impact of  physicians’ electronic access to imaging and test results, 
but not other aspects  of electronic health record (EHR) use.    We did, however, analyze the  relationship of EHRs to test ordering in a subsidiary analysis.  While  physicians use of a full EHR was associated with a 19% increase in image  ordering, as we noted in the paper this finding was not statistically  significant.  
While we cautiously (and properly) interpreted this as a “null”  finding, these data are inconsistent with Mostashari’s optimistic view that use  of a full EHR reduces costs.
He asserts that 
our 2008 data are passe, and that health IT meeting  today’s “meaningful use” criteria definitely saves money. The data we analyzed  were the latest available data when we initiated the study.  While the  proportion of outpatient physicians utilizing health IT has grown since 2008, 
we  are unaware of any “game changing” health IT developments in the past four years  that are would produce substantially different results if the study were  repeated today.  The EHR vendors that dominated the market in 2008 remain, by  and large, today’s market leaders, and their products 
have undergone mostly  modest tweaks.  Mostashari’s contention that 2012  EHRs  – incorporating  decision support and electronic information exchange – save money in ways not  possible in 2008 
should be tested through additional research but remains merely  a hypothesis. We hope that some day his predicted savings can be achieved.
Dr. Mostashari offers his own explanation for our findings, suggesting that  doctors who are inclined to order more tests are also inclined to purchase  health IT for viewing test results electronically rather than on paper.  He  offers no evidence for this assertion and ignores the fact that we explored (and  rejected) this explanation by analyzing subgroups of doctors who are unlikely to  be the decision maker for IT purchases – e.g. employed physicians, those working  in an HMO setting etc.  In other words, electronic access to results predicted  more test ordering whether or not the ordering physician was responsible for  health IT purchases.
He incorrectly states that our analysis did not take into account patients’  severity of illness, physicians’ level of training, and the nature of  physicians’ financial arrangements.  In fact, we reported subsidiary  multivariate analyses that included several serious diagnoses; all of our models  included physician specialty (which we specified in several different ways); and  all models included adjustment for an extensive list of indicators of financial  arrangements (e.g. whether the physician owned the practice or was an employee;  the type of office; whether the practice was owned by a hospital; whether the  physician was a solo practitioner; whether the physician’s compensation was  based, in part or whole on “profiling”; and whether the practice was  predominantly prepaid).  We also performed a series of subsidiary analyses that  explored whether physicians with a proclivity to “self refer” patients for  imaging tests accounted for our finding; they didn’t.
Dr. Mostashari criticizes us for failing to assess whether health IT improved  the quality or appropriateness of care.  
Of course, these were not the topic of  our research.  Those are different studies for a different time.  However, we  would note that 
other large-scale studies have found no, or trivial quality improvements  associated with HIT outside of a few flagship institutions
4-6.
Dr. Mostashari’s strongest claim is that observational studies like ours (and  most other health policy studies, including some by Dr. Mostashari himself)  cannot prove causation. This is surely true.  
As long time teachers of evidence  based medicine we took care to couch our conclusions in cautious terms, stating  only that “Computerization, whatever its other benefits, remains unproven as a  cost control strategy.”But Dr. Motashari is less cautious, 
asserting that the case for HIT is  closed.  The paper he cites to buttress this claim (authored by members of his  own agency) culled studies reporting any impact of HIT on virtually any aspect  of care, 
and accepted authors’ claims of benefit without regard to study quality  or statistical niceties.  Thus, a focus group’s impressions of benefit are  accorded the same weight as nationwide studies of Medicare data showing  virtually no impact of computerization on quality measures.  Reports of a  reduction from 70% to 38 % in “missed billing opportunities” or a $7,000  reduction in office supply costs are among the 92% of studies judged  “positive”.  While the literature review he cites is interesting, nothing in it  contradicts our findings.
Dr. Mostashari is also correct in reiterating that randomized trials are the  best way to assess health IT.  
In fact, no randomized trial has ever been  published that examines patients’ outcomes or costs associated with  off-the-shelf health IT systems that dominate the U.S. market.  No drug or new  medical device could pass FDA review based on such thin evidence as we have on  health IT.  Yet his agency is disbursing $19 billion in federal funds to  stimulate the adoption of this inadequately evaluated technology. 
Dr. Mostashari  is perhaps the only person in our nation who commands the resources needed to  mount a well done randomized controlled trial to fairly assess the impact of  health IT, and the comparative efficacy of the various EHR options.Finally, Dr. Mostashari’s u
nbridled faith in technology is mirrored by his  belief that ACOs are the next panacea for health costs and quality.  That health  policy flavor-of-the-month also remains wholly unproven.