Much has already been written about the Oregon Medicaid study that just came out in the New England Journal of Medicine. Unfortunately, the vast majority is reflex, rather than reflection.  The study seems to serve as a Rorschach test of sorts, confirming people’s biases about whether Medicaid is “good” or “bad”.  The proponents of Medicaid point to all the ways in which Medicaid seems to help those who were enrolled – and the critics point to all the ways in which it didn’t.  But, if we take a step back to read the study carefully and think about what it teaches us, there is a lot to learn.

Here is a brief, and inadequate, summary (you should really read the study):  In 2008, Oregon used a lottery system to give a set of uninsured people access to Medicaid.  This essentially gave Kate Baicker and her colleagues a natural experiment to study the effects of being on Medicaid. Those who won the lottery and gained access were compared to a control group who participated in the lottery but weren’t selected.  Opportunities to conduct such an experiment are rare and represent the gold standard for studying the effect of anything (e.g. Medicaid) on anything (like health outcomes).  Two years after enrollment, Baicker and colleagues examined what happened to people who got Medicaid versus those who remained uninsured.  There are six main findings from the study.  Compared to people who did not receive Medicaid coverage:

1.  People with Medicaid used more healthcare services – more doctor visits, more medications and even a few more ER visits and hospitalizations, though these last two were not statistically significant.
2. People with Medicaid were more likely to get lots of tests – some of them probably good (cholesterol screening, Pap smears, mammograms) and some of them, probably bad (PSA tests).
3. People with Medicaid, therefore, not surprisingly, spent more money on healthcare overall.
4. People with Medicaid were less likely to go bankrupt due to healthcare expenditures.
5. People with Medicaid had less depression and overall, had better health-related quality of life.
6. People with Medicaid did not have meaningful improvements in their hypertension, cholesterol, diabetes, or other measures of overall health.

It is first worth taking a moment to dispense with those who will try to nitpick the methodology.  Read through the paper carefully and spend time going through the 62 page single-spaced supplementary appendix and you’ll find that this is about as good of a study as will be done on this topic for the next generation.  Kate Baicker, who led the study, is the smartest person I know and whenever I disagree with her, it’s because she’s right and I’m wrong.  These are the gold standard of folks using the gold standard of methodology to answer an incredibly important question: what is the effect of Medicaid on financial, mental, and physical health.  So, let’s get to the lessons.

1.  Insurance works.  The goal of my homeowner’s insurance is that if I have a fire, I won’t become bankrupt.  The goal of health insurance should be to ensure that if you get hit by a bus, you won’t go bankrupt.  Medicaid, as insurance, worked.
2. Insurance gives you peace of mind.  I never lie awake worrying that if I get sick, my family will go bankrupt.  Medicaid may therefore be giving people reassurance, and making them feel better, which may be why there was less depression in the Medicaid group (they certainly weren’t taking more anti-depressants).
3. Insurance improves access to healthcare services.  Although people without health insurance still got healthcare (they were spending $3,257 per year on healthcare, seeing 5.5 doctors a year, getting medications, outpatient surgery, etc.), people on Medicaid got more.

But this was all predictable and none of it should be a surprise.  What has been fundamentally misunderstood is why it didn’t lead to better health.  And that is the biggest lesson from Oregon:

Healthcare isn’t health, and the missing link is Quality.

Let’s unpack this.  To date, the notion for improving health has been simple: if we give people access to health insurance, they will get more care, and therefore will have better health.  Oregon tells us that’s not quite right.  The lesson from Oregon is:  if we give people access to health insurance, they will use more healthcare, and they will feel better for it.  But, their health may not be that much better off.  How could this be?

The explanation is simple.  It’s not about access to healthcare; it’s about access to high quality healthcare.  Americans fail to consistently receive high-quality care and the people on Medicaid in Oregon were no exception.  There is some evidence that providers who disproportionately care for Medicaid patients deliver lower quality care, but the problem is much bigger than Medicaid.  In fact, most Americans get pretty mediocre quality healthcare.  Therefore, not surprisingly, healthcare often fails to improve health.

In the Oregon study, we see that many people, especially in the high risk groups, have poorly controlled blood pressure, diabetes, hypercholesterolemia, and depression.  Yes, Medicaid seemed to help a little, but not enough.  People with Medicaid had, on average, 7.2 office visits over the past 12 months.  That’s more than once every 2 months.  Yet, their blood pressure, cholesterol, and blood sugars barely budged.  This is not an access problem.  This is not about “Medicaid is bad” or “insurance is bad”.  This is about what happened (or did not happen) in those visits – namely, evidence-based care that we know improves health.

Most healthcare policymakers talk about the three legs of the stool of the healthcare system: cost, access, and quality.  The Affordable Care Act makes a big effort to improve access, but does less on cost and little on quality.  That’s unfortunate.  Oregon reminds us that if we want to improve the health of the population, we will have to make real and concerted efforts to ensure that people are receiving high-quality care.  We can’t just improve access and think that our job is done – in fact, its just the beginning.

Ultimately, while many factors affect health (such as education, income, neighborhoods where we live, etc.), healthcare matters too.  And it better – we’re spending $2.8 trillion on it.  Oregon tells us that insurance has its benefits – it gives us peace of mind and improves access to health services like office visits and preventive screening.  But it doesn’t do that much for health, because it’s not about access to healthcare.  It’s about access to high quality healthcare.  Quality is the link between healthcare services and better health outcomes.  And we need to spend more time working on that.

Ensuring that Americans who live in rural areas have access to health care has always been a policy priority.  In healthcare, where nearly every policy decision seems contentious and partisan, there has been widespread, bipartisan support for helping providers who work in rural areas.  The hallmark of the policy effort has been the Critical Access Hospital (CAH) program– and new evidence from our latest paper in the Journal of the American Medical Association suggests that our approach needs rethinking.  In our desire to help providers that care for Americans living in rural areas, we may have forgotten a key lesson: it’s not about access to care.  It’s about access to high-quality care.  And on that policy goal, we’re not doing a very good job.

A little background will be helpful.  In the 1980s and 1990s, a large number of rural hospitals closed as the number of people living in rural areas declined and Medicare’s Prospective Payment System made it more difficult for some hospitals to manage their costs.  A series of policy efforts culminated in Congress creating the Critical Access Hospital program as part of the Balanced Budget Act of 1997.  The goals of the program were simple: provide cost-based reimbursement so that hospitals that were in isolated areas could become financially stable and provide “critical access” to the millions of Americans living in these areas.  Congress created specific criteria to receive a CAH designation: hospitals had to have 25 or fewer acute-care beds and had to be at least 35 miles from the nearest facility (or 15 miles if one needed to cross mountains or rivers).  By many accounts, the program was a “success” – rural hospital closures fell as many institutions joined the program.  There was widespread consensus that the program had worked.

Despite this success, there were two important problems in the legislation, and the way it was executed, that laid the groundwork for the difficulties of today.  First, Congress tried to show “flexibility” by allowing states to waive the distance requirement – and waive it they did.  The number of hospitals designated as CAH increased by nearly 50% over the 2000s.  By 2010, there were nearly 1,300 CAHs, many of them in suburbs or even urban areas, just miles down the road from a large institution with no mountains or rivers to impede travel.  Congress finally stepped in to close the loophole, but in many ways, it was too late.  Nearly 1 in 4 acute-care American hospitals today has the “CAH” designation.

The excessive use of the CAH designation may seem to primarily be a money problem.  We know, based on MedPAC data, that hospitals that get the CAH designation get paid substantially more than they would otherwise – and that in the early years of the program, CAH cost growth substantially outpaced comparison hospitals.  Having nearly 1,300 hospitals with the CAH designation is not great for managing cost growth – “cost-based” reimbursement is not the ideal way to incentivize organizations to pay attention to efficiency.  But the problem is much bigger than that – the real issue is that we have very little information about the quality of care that these institutions provide – and that’s the topic of our latest JAMA paper.

The biggest problem with the CAH program is that policymakers consistently exclude these hospitals from national quality improvement initiatives.  All U.S. hospitals have to collect data on process measures and report them to CMS – except the Critical Access Hospitals.  Most U.S. hospitals now have their performance on outcomes measures (such as mortality) or utilization measures (such as readmissions) publicly reported through Hospital Compare.  Critical Access Hospitals?  Not so much.  Even value-based purchasing, the new pay-for-performance program that CMS is running, has mainly exempted CAHs.  The arguments for exempting CAHs are numerous, including the notion that it may be too hard for these organizations to collect and manage quality data, and that they should spend their limited resources elsewhere.  There are some practical issues, of course, such as small sample sizes, but those issues are manageable – and there are plenty of small hospitals with small sample sizes that continue to collect data and report them to CMS.

So, what’s the impact of our efforts to protect these hospitals from the quality initiatives?  The data are coming in, and it’s not good news.  In our latest paper in JAMA, we find that a decade ago, mortality rates for AMI, CHF and pneumonia were pretty comparable at CAHs and non-CAHs (especially when you adjust for things like hospital size, case volume, etc.).  Over the next 9 years, however, there was a clear separation: while most hospitals improved, CAHs got worse.  By 2010, a patient arriving at a CAH with an acute MI had a 33% higher chance of dying than a comparable patient elsewhere.  Even in our matched analysis, where we compared CAHs to other small, rural hospitals that were not in the CAH program – we see similar findings:  CAHs and other small, rural hospitals started off comparable in 2002 – but by 2010, the CAHs were substantially worse than other small rural hospitals.  Other studies, including some of our prior work, have found similar results.

So what’s the take-home?  First and foremost, we have to remember that many of these hospitals do provide critically important access to care for many Americans.  However, the program – built on the best of intentions, needs updating.  There are three things that policymakers ought to do right away.

First – eliminate the Critical Access designation for any hospital that does not meet the original criteria.  In 2010, nearly 17% of CAHs were not in small towns or rural areas. Some are even in urban locations.  They are not “critical access” hospitals in any sense of the notion — and should have their designation revoked (unless they can make a very compelling case otherwise).  Second, these hospitals should be participating in all national quality improvement initiatives.  Even though they may have small sample sizes, the very act of collecting the data and reporting it is likely to be helpful.  The Institute of Medicine said as much back in 2004:  “The committee emphasizes that rural providers should not be excluded from public reporting initiatives. Public disclosure and eventually pay-for-performance payment methods…are potentially powerful incentives for encouraging improvements in quality. Rural providers, like urban, will benefit from these external levers for change.”  Nearly a decade later, there is still time to heed the IOM’s call.

Finally, we need a more proactive approach to helping these hospitals manage their sickest patients.  Hospital care has become extremely complex and the challenges for CAHs will only get steeper the longer we wait.  The defenders of the current program will argue that it is not fair to expect CAHs to manage an acute MI patient as well as a big academic center can.  But if we are being patient-centered, that is the wrong argument.  The right argument is that we can, and should, do better as a health care system for our rural populations.  The question now is – how can we help these hospitals manage their acute MI patients better?  How can we help them manage their pneumonia or stroke patients better? Here, there is a tremendous role for technology and for policy.

Every one of these hospitals should be equipped with telemedicine – and should have a formal partnership with a bigger institution.  Medicare can provide extra payments to larger hospitals that agree to partner – but tie part of those payments to outcomes for CAH patients.  In exchange for extra resources, these larger hospitals should provide technical assistance around quality measurement and improvement.  Further, we need to encourage CAHs to transfer their sickest patients to these partner institutions when additional care is warranted (amazingly, the proportion of patients who are getting transferred from CAHs to other hospitals has fallen by 25% over the past decade) and ensure that once the patient is stabilized, they are transferred back to the CAH.  These partnerships do exist among some subset of Critical Access Hospitals, but clearly not enough.  They need to be universal and a key part of the program.

Over 16 years ago, we embarked on a national effort to save rural hospitals – and closures of rural hospitals have declined precipitously.  Preserving access to hospital care is a good thing.  Yet, the program clearly has also gone astray in important ways:  there are too many hospitals with the CAH designation that do not need it – and it’s wasting taxpayer money.  Even more importantly, our effort to shield these hospitals from the difficulties of participating in national quality improvement efforts may have been well intentioned, but it has not benefited those Americans who count on CAHs for their hospital care.  We can surely do better.

In 2006, Governor Mitt Romney signed Chapter 58 of the Acts of 2006 entitled “An Act Providing Access to Affordable, Quality, Accountable Health Care.”  It has been described by many names, including Massachusetts Healthcare Reform (MHR), Romneycare, or simply, as the template for the Affordable Care Act.  The goal of the act was straightforward: to ensure near-universal access to health insurance for citizens of the Commonwealth of Massachusetts.  The bill quickly led to insurance expansion: by 2010, 94.2% of adults under 65 had health insurance, an 8 percent increase over the 86.6% in 2006.  By all accounts, the goals of insurance expansion were met.

But the bill has not been without controversy.  There have been two main concerns: first, that the bill did too little to control rising healthcare costs.  The cost crisis led to the 2012 bill that many refer to as “Mass Health Reform 2.0” – formally called Chapter 224 of the Acts of 2012.  Its focus is to curtail healthcare spending, and while reasonable people have reasons for skepticism about the likelihood of success, that’s a topic for another day.

The second concern was that bringing hundreds of thousands of new people on to the health insurance rolls without a commensurate increase in physician supply would overwhelm the state’s supply of physicians.  The logic behind the concern was as follows:  health insurance expansion created nearly 400,000 newly insured residents.  As these folks rushed in to see primary care physicians, all the empty spots filled up, the primary care offices got overwhelmed, and access for everyone else was diminished.  Stories of physician shortages abounded: the Massachusetts Medical Society called the shortage of primary care at a “critical level”, citing its own surveys (which were of poor quality).  The Wall Street Journal editorial page ran stories entitled “RomneyCare’s bad outcomes keep coming”, citing the same MMS statistics.

So, for a hypothetical 80 year old woman we will call Ms. Jones, who has congestive heart failure, getting into her PCP was harder.  She used to see her PCP every 2 months, but after Mass Health Reform, had to wait longer between visits.  And, when her breathing worsened one night, instead of getting seen by her PCP the next morning, she had to go to the emergency room and ended up getting admitted.  Indeed, people worried that for the most vulnerable patients, those who rely on primary care to stay out of the hospital, Mass Health Reform decreased access, made their lives worse, and led to unnecessary hospitalizations and worse outcomes.

The concerns over insurance expansion without adequate provider expansion, of course, become that much more salient in the context of the Affordable Care Act.  If Massachusetts, with its large supply of physicians and a reasonably high insured rate prior to health reform, could suffer broad shortages that hurt vulnerable populations, the rest of the nation is surely in trouble.  This was the scenario played out repeatedly in the political debates of 2012 when Mitt Romney was running for the Republican nomination.  Not surprisingly, these discussions were generally data-free.  We felt that empirical input would be helpful.  Whether Massachusetts residents suffered because of the reported shortage of primary care was a serious question that needed to be addressed with real data, and we set out to do so.

To determine if Massachusetts residents were negatively affected, we examined rates of preventable hospitalizations, those that result directly from diminished access to effective primary care, before and after health reform kicked in.  We focused on older adults, the Medicare fee-for-service population, who rely on primary care, hypothesizing that if their access to primary was curtailed, they would be susceptible to these preventable admissions.  We studied Massachusetts from 2005 through 2010, and used the rest of the New England states as controls.  We figured any effect would be particularly pronounced among those over 80 years of age, who might be particularly vulnerable to disruptions.  Finally, we thought that the counties within Massachusetts where the insurance uptake was the greatest, and therefore where the biggest surge of new patients to PCPs might occur, would see the biggest negative effects.

The Impact:  So What Happened?

So what did we find?  Not much.  Our study was well powered to detect even small differences – and we found no negative impact of MHR at all.  In fact, contrary to our hypothesis, rates of preventable hospitalizations fell somewhat more rapidly in Massachusetts after the reform than it did in control states (see Exhibit 3 of the paper in Health Affairs).  The effect was small and whether it was due to the reform or some other factor is unclear.  What is clear is that if Massachusetts Health Reform did lead to a negative spillover on the previously insured, it was not substantial enough to have a deleterious effect on their outcomes.  In every group we examined, Massachusetts improved as rapidly or even more so as the control states.

So are we done worrying about Massachusetts?

Does this mean that we should table the concerns about MHR having a negative effect on the previously insured?  Not quite yet.  It is possible that older, vulnerable patients saw their primary care physicians less often, or maybe they had extra visits to the emergency room and received more tests and procedures as a result of having less time with their PCPs.  These are ongoing analyses and we expect to have answers soon. However, at least in terms of the bottom line, preventable hospitalizations, the events you’d worry about the most if there was restriction on access to primary care, there was no negative effect.

Implications for the Affordable Care Act:

If older Americans in Massachusetts did not experience clinically meaningful harms as a result of insurance expansion, what does this mean for the 49 other states that are about to expand their pools of the insured over the upcoming years?  It is not straightforward to translate the Massachusetts experience to Texas, where a quarter of the population is uninsured and there are fewer primary care physicians per capita.  Might insurance expansion there or in Florida have much bigger effects?  Maybe.  However, our findings suggest that our healthcare system is far more resilient than we think.  Static notions of capacity may be inadequate.  Massachusetts was able to absorb the newly insured population with little disruption and I suspect many other states will as well.

However, maybe in states like Texas and Florida, where the impact of insurance expansion will be more substantial, we could also think more creatively.  Instead of trying to manufacture more primary care physicians, a long and expensive endeavor, we should think harder about how to better use the trained professionals we have.  We need to rethink our “scope of practice” rules that limit the ability of well-trained Physician Assistants and Nurse Practitioners from caring for patients.  We need to think about how to use health information technology more creatively.  Virtual visits and telehealth can make current providers more productive, allowing them to care for more patients by more effectively triaging who can be seen virtually and who needs to be seen in person.  Again, reimbursement and regulatory hurdles slow us down, but if state and federal policymakers are smart, we can fix these issues.

Finally, and most importantly, we need to carefully monitor how insurance expansion plays out in the 49 other states and the District of Columbia.  We need clear metrics to track not just the impact on the previously uninsured (whether they took up insurance or not) but also on the previously insured.  The truth is, while we may split people into the uninsured and the insured, we are all part of the same healthcare system – and what happens to one group likely affects all of us.

Should we hold hospitals accountable for what happens after a patient leaves the hospitals’ doors?  A year ago, I thought the answer was no.  A hospital’s job was to take care of sick patients, make them better and send them on their way.  With more thought and consideration, I have come to conclude that I was probably wrong.  It may be perfectly reasonable to hold hospitals accountable for care beyond their walls, but we should be clear why we’re doing it.   Readmissions are not a good quality measure – but they may be a very good way to change the notion of accountability within the healthcare delivery system.

The debate around the readmissions measure has come to the forefront because of the CMS Hospital Readmission Reduction Program, which penalizes hospitals for “greater than expected” readmission rates.  It has raised the question — does a hospital’s 30-day readmission rate measure the “quality of care” it provides?  Over the last three years, the evidence has come in, and to my read, it is unequivocal.  By most standards, the readmissions metric fails as a quality measure.

Why do I say that readmissions are a poor measure of hospital quality?  First, we have to begin by thinking about what makes a good quality measure.  Quality is about the essence of the thing being produced – a good or a service.  The job of a car is to get you from place A to place B and a high-quality car may be one that does the job reliably, safely, or maybe even comfortably.  The job of a restaurant is to provide a meal that you don’t have to cook — and a high quality restaurant may provide food that is fresh, tasty, or with an attention to service that you enjoy.  What is the job of a hospital?  When you get sick and require hospital services, a high-quality hospital should give you the right treatments, attend to your needs while you’re there, and make sure nothing bad (i.e. a new nosocomial infection) happens along the way.  That’s how we measure hospital quality.

Quality measures for healthcare come in three flavors – structural measures (do you have enough intensivists manning your ICU?), processes measures (did you give the heart attack patient his or her aspirin?) and outcomes measures (did the pneumonia patient die?).  The elemental part of both structural measures and process measures is that they have to be tied to an outcome we care about.  If having more intensivists in the ICU does not lead to lower ICU mortality (or lower complication rates), we wouldn’t think it’s a particularly good quality measure.  We know that giving aspirin to heart attack patients lowers their chances of dying by 25%.  We have multiple randomized trials.  We don’t need much more evidence.  Hospitals that have the right structures in place and reliably deliver the right treatments can reasonably be called high-quality hospitals

What about outcomes measures? It gets trickier.  An “outcome” is a health state – and the goal of healthcare is to maximize the health of the patient.  Death is clearly a bad outcome (with the caveat that for someone with a terminal illness, death may be an expected outcome).  Nosocomial infections are bad outcomes.  Is being readmitted a bad outcome?  It’s a little funny – hospitalization or re-hospitalization is a not a health state.  It’s a clinical service provided to people who are sick.  While no one likes to be hospitalized or readmitted, being admitted doesn’t make you sick.  Being sick gets you admitted (or readmitted).  If we had a sick patient in the ER who was last discharged from the hospital 28 days prior, we don’t make them better by sending them home instead of admitting them.  Yes, sending them home avoids a readmission – but the goal is not to avoid readmission, the goal is to make people better.

When we develop “quality” measures, we also look for external validity.  We know that there are terrific hospitals and mediocre ones.  Arguably, the gold standard for measuring hospital quality is condition-specific mortality rates.  Almost by definition, good hospitals are those that have low risk-adjusted mortality rates while bad ones have high risk-adjusted mortality rates.  So when we consider other potential quality measures, we may begin by asking if they are correlated with the gold standard – mortality rates.  Do process measures have external validity (i.e. do hospitals that give aspirin to their heart attack patients tend to have better mortality?).  We know that they do and although the relationship between process measures and the gold standard outcome (mortality) is not very strong, it adds one piece of evidence that process measures may be picking up something good.  Quality.  What about patient experience scores?  You could argue that higher satisfaction is a health state unto itself and doesn’t need to have external validation – and that’s a reasonable argument.  Further, we do know from studies that hospitals with better patient experience scores tend to have better mortality rates.  No one thinks that patients are happier at these hospitals because these hospitals are keeping patients alive.  Instead, these hospitals are generally well run, and therefore, have better performance across the board.

So if one measure of quality is external validity – being at least somewhat correlated with the gold standard (mortality rates) — how does the readmission measure do?  In a paper published recently in JAMA, we see that readmission rates don’t do so well at all.  Readmission rates are un-correlated with mortality rates.  In fact, for one of the three conditions, the readmission rate seems to go the wrong way:  the best hospitals for heart failure (i.e. those with the lowest mortality rates) have readmission rates that are actually higher.  Not perfect.  Readmissions seem to have little external validity as a quality measure.  Readmissions are, however, correlated with two things:  how sick your patients are, and how poor your patients are.  We now have good data that the Hospital Readmission Reduction Program disproportionately penalizes big academic teaching hospitals (that care for the sickest patients) and safety-net hospitals (that care for the poorest). See table 1 below.

So, given its poor test characteristics, can we justify using the current hospital readmissions measure to grade hospitals on quality?  I don’t think we can.  However, here’s where my own ideas have evolved (OK – another way of saying that I was probably short-sighted in the past), the 30-day readmission measure may be a good way to promote accountability in healthcare.  Let’s think about what that might mean.

Right now, in our very fragmented healthcare system, no one seems accountable for what happens to the patient after they leave.  No one is responsible for ensuring that patients get good follow-up or even that someone calls the patients a few days after they are discharged to make sure they are doing well.  There are probably a dozen other things that some creative entrepreneur can come up with to make sure that patients who leave the hospital do well, recover quickly, and don’t get sick again in a way that requires them having to come back to the hospital.  Unfortunately, our system generally doesn’t reward any of that.  No one gets rich keeping patients healthy and well (and therefore, not needing to be hospitalized).  The Accountable Care Organization (ACO) program begins to do that, and I am hopeful it will make a difference.  But the truth is that most patients are not in an ACO and won’t be anytime soon.

In conversations with colleagues and friends, the readmissions penalty program seems to have gotten some hospitals to think outside of their four walls.   Hospital leadership has started to rethink the role of the hospital.  Hospitals are building relationships with community-based organizations.  Some are creating follow-up clinics while others are calling all the patients who are discharged to make sure they are doing OK at home.  Of course, some hospitals are taking the opposite approach – admitting more patients to “observation” status (so they don’t get penalized for readmissions), a move that saddles many patients with extra healthcare bills.  Others are even sending patients home from the ER if they are in the 30 day window when clinically, they should have been admitted.  This is the consequence of using a measure that is very narrowly tailored.  If the patient is sent home and dies, the hospital will be penalized much less than if the patient is readmitted.  Not very patient-centered.

Getting hospitals to change their business model – to start thinking about what happens to a patient when they head out the door – is a good thing.  But if we’re asking hospitals to change, and be responsible for everything that happens in the first 30 days after discharge, then we should rethink how we pay hospitals.  Here is the promise of “bundled payments” and personally, I’d be much happier if we just bundled all the services that a patient receives after discharge into the initial hospitalization.  We should also make sure that we pay hospitals that care for the sickest patients a lot more, because they will need a lot more post-discharge care.  Finally, we should measure the quality of care that patients receive in those days after discharge.  If CMS wants to use the readmissions program as a way to rethink the hospital episode – I’m all for it.  But, it should be far more comprehensive than whether the patient was readmitted to the hospital within 30 days or not.

The readmissions program seems to be, for some hospitals, having a positive effect. Will it pay off?  Will we see a real, sustained change in the way they provide care to patients after they are discharged?  I hope so.  But remember – some of the best hospitals in America have the highest readmission rates, almost surely because they care for sicker, poorer patients.  See table 2 below.  In the current business model, they are doing things right – taking good care of the patient while the patient is in the hospital.  It’s fine to ask these hospitals to change their business model and to become accountable for what happens to their patients after they are discharged.  But, let’s not call them bad hospitals or suggest that they are providing poor quality care.  There is no evidence that they are.

Table 1.  Hospital Characteristics by Penalty Group from the Hospital Readmissions Reduction Program*

*From Joynt and Jha, Characteristics of Hospitals Receiving Penalties under the Hospital Readmissions Readuction Program, JAMA 2013; 309(4) 342-343

Table 2.  U.S. News Top 50 hospitals* on readmissions and mortality rates

*Top 50 in cardiovascular or pulmonary disease

Over the past decade, there has been yet another debate about whether pay-for-performance, the notion that the amount you get paid is tied to some measure of how you perform, “works” or not.  It’s a silly debate, with proponents pointing to the logic that “you get what you pay for” and critics arguing that the evidence is not very encouraging.  Both sides are right.

In really simple terms, pay-for-performance, or P4P, can be thought about in two buckets:  the “pay” part (how much money is at stake) and the “performance” part (what are we paying for?).  So, in this light, the proponents of P4P are right:  you get what you pay for.  The U.S. healthcare system has had a grand experiment with P4P:  we currently pay based on volume of care and guess what?  We get a lot of volume. Or, thinking about those two buckets, the current fee-for-service structure puts essentially 100% of the payments at risk (pay) and the performance part is simple:  how much stuff can you do?  When you put 100% of payments at risk and the performance measure is “stuff”, we end up with a healthcare system that does a tremendous amount of stuff to patients, whether they need it or not.

Against these incentives, new P4P programs have come in to alter the landscape.  They suggest putting as much as 1% (though functionally much less than that) on a series of process measures.  So, in this new world, 99%+ of the incentives are to do “stuff” to patients and a little less than 1% of the incentives are focused on adherence to “evidence-based care” (though the measures are often not very evidence-based, but let’s not get caught up in trivial details).  There are other efforts that are even weaker.  None of them seem to be working and the critics of P4P have seized on their failure, calling the entire approach of tying incentives to performance misguided.

The debate has been heightened by the new national “value-based purchasing” program that Congress authorized as part of the Affordable Care Act.  Based on the best of intentions, Congress asked Medicare to run a program where 1% of a hospital’s payments (rising to 2% over several years) is tied to a series of process measures, patient experience measures, and eventually, mortality rates and efficiency measures.  We tried a version of this for six years (the Premier Hospital Quality Incentives Demonstration) and it didn’t work.  We will try again, with modest tweaks and changes.   I really hope it improves patient outcomes, though one can understand why the skeptics aren’t convinced.

So what to do?  In a recent issue of JAMA, I outline three principles that are really simple, not all that original or creative, and may be one way to think about correctly structuring P4P programs.  First – focus on the “pay” part – if you really want hospitals and other provider organizations to change behavior, put real money at risk.  I know that large incentives can have the perverse effect of reducing internal motivation, but that primarily happens to human beings (who have internal motivation), not organizations.  In this case, organizations and corporations are not people.  Large organizations focus primarily on incentives.  If the incentives for meeting a performance goal are small, organizations will make small changes.  Their Chief Quality Officer might put it on his “to do” list.  If the incentives are large enough, it will get the attention of the CEO, who will make it her mission to get it done.  Size of incentives matter.

Second, get the right metrics.  Here, I think that we have to stop playing around with process measures.  P4P programs can be way too prescriptive, and focusing on a small number of processes, no matter how “evidence-based” they might be, is not going to get us where we want to be.  We need to focus on a small set of high value outcomes. Who choses?  In the ideal world, if patients actually influenced the healthcare system, providers would figure out what mattered to patients.  Right now, the payers (government through CMS, private insurance companies) get to choose and I think they should focus on what likely matters most to patients.  When patients are hospitalized, they generally prioritize walking out alive, not picking up a new infection along the way, and being treated with respect.  Those sound like good metrics.  Patients would also like, after they are discharged, to not come back to the hospital soon, though I suspect that that’s a lower priority than being alive.

Finally, we need transparency in the way we structure the incentives.  Many of the P4P programs to date have been un-necessarily complicated.  The VBP program, for instance, is quite complex.  For instance, on patient experience measures, your financial reward depends on a combination of achievement (how well did you do), improvement (how much have you gotten better) and persistence (how often did you do well across a range of measures).  For most hospitals, it’s very hard for them to know how well they will do.  My take is a simpler approach:  pick a goal (let’s say the 90^th percentile of performance across the nation) and then, set up a simple scheme.  The closer you are to the goal, the bigger your payments.  So, if the best (90^th percentile) hospitals have a mortality rate for pneumonia of 12%, then hospitals that are at 12.2% will get paid more than the hospital at 13% who will get paid more than the hospital at 14%.  I know it sounds too simple – but it makes sense, avoids game playing, and rewards hospitals purely on performance.

At the end of the day, P4P has to be a tool we use to drive improvements in care.  It’s intuitive, and we already have it in healthcare:  we pay more to doctors and hospitals who do more stuff.  It’s time to pay more for providers that achieve better outcomes.  And the key to success?  Don’t be overly prescriptive about the details of what people should do.  Focus on high level metrics (outcomes), put real money on the table, and then, get out of the way and let providers innovate.  How low can they drive infection rates?  Let’s find out.  Let’s make sure there’s enough money for providers and hospitals to innovate the way they deliver care so that they can do well when they do good.

Healthcare: The Journal of Delivery Science and Innovation, a new journal promoting cutting edge research on innovation in health care delivery, has launched. The questions is, do we really need yet another journal? The short answer is yes. The longer answer is, absolutely yes. Here’s why.

The Need for New Knowledge on Healthcare Delivery

There is an urgent need to improve our mess of a health care system. Healthcare will consume about $2.8 trillion in 2012 – that’s an astronomical amount of money.  To think of it in another way:  spending in Intensive Care Units will make up 1% of all economic activity in the U.S.  In a broader context, about 1 in 5 dollars in the economy will be spent on healthcare.

How will we actually spend the $2.8 trillion? Over a million doctors and nurses will see patients in hundreds of thousands of clinics, hospitals, nursing homes, and countless other settings.  They will see patients who are sick and suffering and will make decisions about how to help them get better.  These intensely personal decisions will be made in the context of a broader healthcare delivery system that is mindboggling diverse, complex, and fundamentally broken.  We are probably wasting more on healthcare than we are spending on education.  Yet, despite all this money and excess (or may be because of it), tens of thousands of Americans are dying each year because of poor quality, unsafe care.  We can do so much better.

Despite all of the attention on healthcare, we still don’t know how to make the system work better.  There are lots of good ideas and no shortage of smart people working on these issues.  We need a venue to see the smartest of these ideas get out.  We need a venue to see bright ideas about policy efforts that might make a difference, or delivery innovation that worked really well.

The Gap in the Existing Healthcare Journal Landscape

Of course, one might ask – aren’t there already venues for good ideas to be disseminated?  The answer of course is yes – but there are not nearly enough.  Not even close.

For a system as complex and messy as ours, the amount of learning we need to do to improve care is immense.  There are terrific journals like New England Journal of Medicine, JAMA, Annals of Internal Medicine, Archives of Internal Medicine and a few others.  They publish high impact research in this area, but they are general medical journals.  They can only dedicate a small amount of space to strategies that improve healthcare delivery.  They need to balance a study on how health IT can reduce medical errors with a randomized, controlled trial of the latest drugs that might change clinical practice.  The number of slots in these journals for high quality work on healthcare delivery is small.

On the other end, there is a journal like Health Affairs.  They are firmly in the health policy space, and have been tremendously impactful.  However, one journal, publishing monthly, can only cover so much ground and they have to leave so much good work behind.  Further, by focusing primarily on policy, they spend a lot less space on innovations in healthcare delivery.  There are others, such as the  BMJ Quality and Safety journal, which also publish important studies, but these typically have a more international focus.

Our goal is not to compete with these journals. We hope to become another important venue for high quality work, with a particular focus on delivery innovation.  Do you have a new way to improve population health?  Tell us about it.  Do you think a quality measure that we’re using is not improving healthcare delivery?  Show us the data.  Ultimately, we want to be a venue for authors as they consider innovative ways to tackle the incredibly complex healthcare delivery system.  We’re looking for data on solutions, but also opinions and synthesis of good ideas.

Our Promise

Why should you send your work to us?  We have three promises:

• First, we will treat you with respect.  We will review your paper quickly and make a decision quickly.  If we make a commitment to taking a paper, we will work with you to make it as good as possible.  We will not ask you to make multiple sets of revisions and then reject it.
• Second, we will publish your work quickly.  In its initial form we’ll be a quarterly journal and we’re still figuring out a web strategy.  However, our hope is that good work gets out there quickly in time for it to have an impact.  That’s what authors care about – and that’s what we care about.
• Third, we will help you disseminate your work.  We not only have a great editorial advisory board, but we have relationships with key clinical leaders and policy makers.  If we think your paper is important enough to publish, then we think it’s important enough for us to email a copy to a key person who could benefit from your insights.  We are dedicated to ensuring that your work has impact.

One of my favorite quotations is one attributed to Vaclev Havel, the visionary and late President of the Czech Republic.  “Keep the company of those who seek the truth.  Run from those who have found it.”  We don’t know the truth about how we improve the healthcare delivery system.  We are seeking the path.  It’s a journey where will need everyone to play a role – clinicians, policymakers, clinical leaders, researchers, and yes, journals.  We hope to play an important role in that journey, facilitating the dissemination of good ideas, tested rigorously.  We hope you’ll give our journal a try and always let us know how we can do our work better.

Ashish K. Jha will serve as a Senior Editor-in-Chief for Healthcare: The Journal of Delivery Science and Innovation. Submission is open now, and the first issue will be released in late spring 2013.

Yesterday’s New York Times headline read that “Medicare Is Faulted on Shift to Electronic Records.”  The story describes an Office of Inspector General (OIG) report, released November 29, 2012, that faults the Centers for Medicare and Medicaid Services (CMS) for not providing adequate oversight of the Meaningful Use incentive program. Going after “waste, fraud, and abuse” always makes good headlines, but in this case, the story is not so simple.

For those not intimately familiar with the CMS policy, in 2009, Congress passed the Health Information Technology for Economic and Clinical Health (HITECH) Act.  The program, administered through CMS and state Medicaid programs, created financial incentives for doctors (and other eligible professionals) and hospitals to adopt and “meaningfully use” a certified electronic health record (EHR).  To receive financial incentives, which began to be paid in May 2011, doctors and hospitals “attest” that they have met the meaningful use requirements, providing an affirmation for which they are held legally accountable.

The process works as follows: health care providers visit a CMS website, register, and enter data demonstrating that their EHRs are “certified” and that they met each of the individual requirements for meaningful use. Then they attest that that all the data they entered is true.  For example, a physician might have to report, to meet just one of the 20 meaningful use measures, how many prescriptions she wrote over the past 90 days, and how many she wrote electronically.  My conversations with colleagues suggest that it can take a lot of time for providers to gather all the data they need to “attest” to meeting Meaningful Use.  Then, CMS runs logic checks to ensure that the numbers entered make sense and, if there are no errors, they cut the provider a check. Through September, 2012, CMS paid out about $4 billion in incentives to 82,000 professionals and more than 1,400 hospitals.

What The Office Of Inspector General Recommended …

The Office of Inspector General examined how carefully CMS is overseeing this program, with particular interest given the reliance on self-reporting of meaningful use, and found areas for improvement.  Several of the OIG recommendations for improving oversight are sensible, measured, and very likely to improve the integrity of the program.  For instance, OIG recommends that CMS provide detailed guidance to providers about what constitutes adequate documentation to support their attestation.  This is the equivalent of the IRS providing guidance on what documentation you need to prove that your tax deductions are legitimate.  Another reasonable OIG recommendation is that certified EHRs be able to produce automated reports about all the functions required to meet meaningful use.  I suspect this will not be particularly onerous for EHR vendors to meet.

… And Where OIG Went Wrong

Where the OIG goes astray is their recommendation that CMS “obtain and review supporting documentation” from selected doctors and hospitals prior to payment.  This is the equivalent of the IRS asking a large chunk of Americans to send in their receipts and detailed explanations along with their 1090s before they get their refund.  Based on the screening tool discussed in the OIG report, about 100,000 physicians and 800 hospitals would be subject to creating these detailed reports with a large amount of supporting evidence each year — and CMS would need to add a substantial number of staff to review all these reports before making any payments.

CMS chose not to concur with this recommendation, and I think CMS is right. There is little evidence to date of any fraud, waste, or abuse in the EHR incentive program.  Were they to follow this OIG recommendation, CMS would effectively make waste in the program more likely.

This is not to say that there are not areas in which CMS should be more aggressive in their oversight.  Indeed, CMS pays out more than $500 billion in taxpayer money each year for medical care, and there is ample evidence that a substantial proportion of that money goes to health care services that harm our nation’s elderly while providing little clinical benefit — that’s the real “waste, fraud, and abuse.” We continue to pay billions for poor quality and unsafe care.  CMS needs to develop more sophisticated and nuanced approaches to ensuring that it pays more for better care, pays less for poor quality care, and protects not only the taxpayer dollar, but also the health of Medicare beneficiaries.  That’s worth getting aggressive about.

CMS’s Approach On EHR Payments Strikes The Right Balance

Given that there is no evidence that doctors and hospitals are systematically committing federal fraud by reporting that they are meaningfully using EHRs when they are not, CMS has instead planned a post-payment audit.  And if physicians or hospital executives are found to have deliberately and consciously lied in order to get incentives, they should be prosecuted.  If they made an honest mistake — and given the complexity of meaningful use criteria, this is a real possibility — they should give the money back and potentially pay a fine.  But creating a huge new burden will dissuade many providers from even adopting EHRs, and continuing to rely on paper records is no way to deliver health care.  The cost of the latter to the American public, in terms of duplicate tests, medical errors, and general poor quality care, is far more substantial.

In each regulatory decision, there is a balancing act:  have too few checks and there will be widespread fraud; be overly heavy-handed and you may end up penny-wise and pound-foolish.  The approach that CMS seems to be taking is, in the famous words of President Reagan, “trust, but verify.”  Trust that providers are being honest – but verify through selected audits.  It appears to get the balance right. This approach was good enough for President Reagan’s dealings with the Soviet Union and I suspect that it’s good enough for CMS’s dealings with doctors and hospitals.

I’ve been getting emails about the New York Times piece and my quotation that the penalties for readmissions are “crazy.”  It’s worth thinking about why the ACA gets hospital penalties on readmissions wrong, what we might do to fix it—and where our priorities should be.

A year ago, on a Saturday morning, I saw Mr. “Johnson,” who was in the hospital with a pneumonia.  He was still breathing hard but tried to convince me that he was “better” and ready to go home.  I looked at his oxygenation level, which was borderline, and suggested he needed another couple of days in the hospital.  He looked crestfallen.  After a little prodding, he told me why he was anxious to go home: his son, who had been serving in the Army in Afghanistan, was visiting for the weekend.  He hadn’t seen his son in a year and probably wouldn’t again for another year.  Mr. Johnson wanted to spend the weekend with his kid.

I remember sitting at his bedside, worrying that if we sent him home, there was a good chance he would need to come back.  Despite my worries, I knew I needed to do what was right by him.  I made clear that although he was not ready to go home, I was willing to send him home if we could make a deal.  He would have to call me multiple times over the weekend and be seen by someone on Monday.  Because it was Saturday, it was hard to arrange all the services he needed, but I got him a tank of oxygen to go home with, changed his antibiotics so he could be on an oral regimen (as opposed to IV) and arranged a Monday morning follow-up.  I also gave him my cell number and told him to call me regularly.

Much of the weekend went smoothly.  When I talked to him on Sunday morning, he reported having slept poorly but had a joyful tone in his voice that I never heard in the hospital.  He was planning on having a few beers with his son and watching the Patriots game.  I told him to take it easy on the beers.

Sunday afternoon, I caught him during half-time and he assured me everything was fine.

On Monday morning, I got a call that Mr. Johnson was back in the hospital. I rushed to his room to see him lying in bed, looking sad.  He told me that his breathing had gotten worse overnight and at 3 a.m. his son drove him to the hospital.  His vital signs looked fine, although his oxygenation was a little worse than Saturday.  He screwed up, he said, and told me I’d been right.  He should not have gone home.  I asked if he had enjoyed the weekend, and his face lit up.  He had loved it.  Let’s be clear: he had been right to go home. There was no screwup.  We had gotten him a weekend at home with his son, who would soon be heading back to Afghanistan.

In 2012, more than 100,000 Americans will die in U.S. hospitals because of medical errors such as preventable infections, receiving the wrong drug, or having the wrong surgery.  Even more Americans will likely die because they failed to get simple therapies like the right antibiotic for their pneumonia.  Millions of people will report suffering in the hospital from undertreated pain or the indignities of not being always treated with respect.  Yet the Affordable Care Act says that my “mistreatment” of Mr. Johnson—sending him home and having him come back—was far more egregious and deserves the biggest penalties.  While the ACA is extremely important in improving access to millions of Americans, several of the provisions to improve the “delivery system” are not quite right.  The notion that readmitting people to the hospital is worse than killing them due to medical errors?  Sorry, but that is crazy.

The Leapfrog Group will soon be putting out another report of patient safety in U.S. hospitals (I’m on their advisory panel).  It will provide letter grades on the state of safety of every hospital.  The grading system is not perfect—primarily because hospitals are not required to report their rates of medical errors. Yet Leapfrog soldiers on, trying to make their best assessment.  I wish Medicare would make patient safety half as much of a priority as reducing readmissions.  Oh, and by the way?  Reducing medical errors can likely save us a lot more money than reducing readmissions—so even if we do it for the money, that should be our target.

So—should we penalize hospitals for readmissions?  I think it’s probably fine (although we should know that we will primarily end up penalizing hospitals that care for the sickest and poorest patients).  But by putting so much energy on readmissions and so little on patient safety, we have made our priorities clear, and I think they are the wrong priorities.

If my hospital had made my readmission rate part of my performance evaluation, would I have sent Mr. Johnson home that weekend? Maybe not.  I could have easily strong-armed him into staying, and he would have listened.  He was, what we call, a “compliant” patient.  But if we had kept him in the hospital, he would have lost the chance to watch the Pats game with his son.  His son and family would have lost having the weekend with their dad and husband.  But I would have “won,” coming across as a better doctor for having a lower readmission rate.

Policies have consequences.  They set up subtle, often perverse incentives.  Before we decide that readmissions are the biggest priority for cleaning up American hospitals, we should ask whether Mr. Johnson should have been sent home that weekend.

Currently, India spends about $20 per person per year on healthcare and spending more once seemed like a peripheral concern, taking a back seat to basics like food and sanitation.  However, in the past decade, as the Indian economy has grown and wealth followed, Indians are increasingly demanding access to “high quality” healthcare.  But what does “high quality” mean for a country where a large proportion of the population still goes hungry?  Where access to sanitation is so spotty that the Supreme Court recently had to decree that every school should have a toilet?  What is “high quality” in a setting where so many basics have not been met?

It turns out that “high quality” may mean quite a lot, especially for the poor.  A few weeks ago I spent time in Delhi, meeting with the leadership of the Indian health ministry.  I talked to directors of new public medical schools and hospitals opening up around the country and I met with clinicians and healthcare administrators at both private and public hospitals.  An agenda focused on quality rang true with them in a way that surprised me.

The broad consensus among global health policy experts is that countries like India should focus on improving “access” to healthcare while high income countries can afford to focus on the “quality” of that care.  The argument goes that when the population doesn’t have access to basic healthcare, you don’t have the luxury to focus on quality.  This distinction between access and quality never made sense to me.  When I was a kid in Madhubani, a small town in in the poor state of Bihar, I remember the widespread impressions of our community hospital.  It was a state-run institution that my uncle, a physician, once described as a place where “you dare not go, because no one comes out alive”.  His point was that the quality was so awful that patients were routinely injured or even killed by medical errors.  People avoided it until they were at death’s door.  For the people of Madhubani, was this an “access” problem or a “quality” problem?  It was clearly both and the split between access and quality also made no sense to them.

The Indian parliament is now debating its 12^th 5-year plan (yes, the government still relies on Soviet-style 5-year plans) which would increase government spending on healthcare from 1% of GDP to 1.6%, a sizable amount (although I think they should be spending a lot more).  Much of this new spending is focused on access: ensuring “universal health coverage” and strengthening primary care. These are good things. However, at least some part of the spending should be dedicated to measuring quality.   A basic information infrastructure that tracks not just who received care but also the kind of care they received and if it did them any good is vital. Without answers to these questions, we will never know how much of the new investment improved health and how much was wasted.

Measuring quality has to be a part of a broader strategy that closely tracks health of the population.  Health outcomes measures will allow policymakers to decide when additional resources need to go into public health, when they need to be spent on direct healthcare (i.e. improving access) and when they need to be spent ensuring that the care is good (i.e. quality).  Measuring health outcomes and making those data widely available to the public will reduce the risk that the extra spending is wasted by a corrupt political system.  It will hold policymakers accountable to the bottom line:  improving the health and well-being of the population.

Two decades ago, Americans believed that having a licensed physician care for you at an accredited hospital assured quality.  Yet years of relentless data suggesting otherwise has changed the way we think.  We now know that an accredited hospital with licensed physicians can have mortality rates four times higher than another hospital across the street.  This variation has a huge cost, leading to unnecessary deaths and wasted resources.  Regulation, while a prerequisite, does not assure high value healthcare.  That is the most important lesson for Indian healthcare.  If India wants a robust healthcare system that delivers high quality healthcare at a reasonable cost, it needs a quality strategy with robust measurement of processes, outcomes and costs.  This kind of approach is necessary for any society that does not wish to pour large sums of money into wasted care.

My previous question about how we can ask focus on quality when food and sanitation are still not always available, presents us with false choices.  The failure to get clean water to every household is not one of resources, but of effective governance.  Poor quality schools would certainly be made better with more spending, but would be dramatically improved with greater accountability.  The notion that India doesn’t have the resources to measure and ensure quality actually creates a huge risk.  If India goes down the road of widespread healthcare access expansion without concurrently focusing on quality, it will likely waste an immense amount of resources and lives.  For the U.S., this has meant hundreds of billions of dollars of waste and tens of thousands of unnecessary deaths each year.  As India thinks about making greater investments in healthcare, the question cannot be whether it can afford to measure and ensure high quality care.  The real question is – can it afford not to?

A few months back, we admitted a patient we’ll call Mr. Jones to the hospital for a severe gastrointestinal bleed.  We had discharged him two weeks earlier after he had come in with a heart attack and made sure he was on aspirin to prevent future cardiac events.  He dutifully took his aspirin and on the day of the readmission, had a massive bleed.  He made it to the hospital barely alive and an endoscopy in the ICU showed an active bleeding gastric ulcer.  For Mr. Jones, the gastrointestinal bleed, likely brought on by the aspirin, was an “unintended consequence” that almost killed him. Yet no one questioned whether we should have given him aspirin in the first place.  I felt terrible about what had happened but found solace in knowing that while for some patients the risks of aspirin are worse than the benefits, for the general population of people like Mr. Jones, the benefits are clearly worth the side-effects.

We do risk-benefit analyses every day in clinical care, knowing that for some patients, the benefits will be outweighed by the harm.  We try to be thoughtful about who might be hurt or not, but most of the time, we just can’t predict.  So, when the benefits appear to outweigh the risks, we move forward and try to learn from cases like Mr. Jones.

While this kind of risk-benefit analysis is common in clinical practice, it’s unfortunately not how we discuss health policy interventions.  No policy intervention is ever without risks, and it is rare that a new policy will have no side-effects at all.  Yet, every time policymakers put in a new initiative, they sell it as a panacea. Critics, upon finding an unintended consequence, then declare the whole thing a failure.

An excellent example of this is health information technology, a topic that I have blogged about in the past (here and here).  Proponents only talk about its benefits, allowing critics to highlight every shortcoming and failure.  Thank goodness I don’t have to deal with proponents and critics like that every time I consider prescribing aspirin to my patients.

From this backdrop comes an important new JAMA study by Dr. Karen Joynt (on which I was senior author).  We examined what happens to patients with an acute myocardial infarction in states that have public reporting of outcomes for patients undergoing percutaneous coronary intervention (PCI, better known as angioplasty and stenting).  Three states have instituted mandatory public reporting of outcomes for this procedure.

Proponents of public reporting argue that transparency will motivate poor performers to improve and provide information that consumers can use to choose the best providers. However, it was not without controversy.  The critics voiced concern that public reporting would lead to denial of care for the sickest patients because doing so could make physicians’ mortality statistics look worse.  Whether public reporting leads to better care, unintended consequences, or both is an empirical question – a question our research team wanted to answer.

So what was the punch line?  We found that public reporting was associated with reduced access to PCI for patients who had a heart attack.  Stated simply, in states with public reporting of angioplasty outcomes, heart attack patients were less likely to receive this life-saving procedure.  This was especially true among the sickest patients (the ones who probably benefit the most).  Even more concerning, there was a trend towards higher mortality for heart attack patients in public reporting states, presumably because they were less likely to receive this life saving therapy (this effect was small and not consistently statistically significant).  This was clearly an unintended consequence, and a bad one at that.

Transparency – making outcomes public so patients have access to information – clearly has its upsides as well.  There’s evidence that public reporting of PCI has led to lower procedure complication rates, presumably because providers are being more careful.  There’s evidence from cardiac surgery that public reporting pushed some of the worst performing surgeons to stop doing surgery altogether.  While difficult to measure, making the healthcare system more transparent to patients has its own benefits in terms of increased trust.

The lesson to draw from our paper in JAMA is that no policy, no matter how well intentioned, is without side effects.  Our job is to think about how to minimize those side effects.  We could improve risk-adjustment methods so that cardiologists get more credit for performing PCI on the sickest patients.  We could carve out some exceptions from public reporting, as Massachusetts has been trying to do, as a response to this kind of data.  Or, ultimately, we can hope that over time, physicians get used to public reporting and simply do what is right for their patients.

When aspirin was first tested in heart attack patients nearly two decades ago, large randomized, controlled trials were conducted, giving us precise estimates of both the benefits and harm.  We could make thoughtful clinical decisions about whether it was worth it or not and in whom.  Most health policy interventions are not tested rigorously before they are implemented.  That’s why we need robust health policy research after policies go into effect.  We need these studies not to discover whether there are any unintended consequences – there usually are – but rather, to quantify the benefit and the downsides so that we as a society can make thoughtful decisions about whether the policies are worth it or not.  Despite the fact that Mr. Jones had a horrible complication from aspirin, I will continue to prescribe it to my heart attack patients.  Despite the fact that transparency seems to reduce access to PCI for critically ill patients, I will continue to advocate for it in healthcare.  The question we now need to ask is how can we do it in ways that lets us maximize the benefit and minimize the harm.