As Medicare Moves Toward Pay-for-Performance, Study Highlights Need for Better Data

BY Lea Winerman  October 18, 2010 at 4:03 PM EDT

Much of the coverage of the health care reform law in its early stages has focused on efforts to expand health insurance coverage. But the law has another focus as well — improving the quality and value of medical care.

Flickr Creative Commons/BestInPlastics

On its health reform Web site, HealthCare.gov, the federal government links to a database called Hospital Compare, a compendium of Medicare data on hospital quality intended to help patients choose the best hospitals for their heart bypass, pacemaker implantation, or other surgery.

But a study published Monday in the journal Archives of Surgery suggests that much of the data available on the “Hospital Compare” site may not help patients make better decisions about where to go for surgery after all. The study found that the “process measures” used on Hospital Compare — things like whether patients receive antibiotics at the correct time before a surgery — don’t correlate with mortality rates at the hospitals, or complications like surgical site infections.

Lauren Nicholas, the lead author of the study and a health economist at the University of Michigan, and her colleagues analyzed Medicare claims for more than 325,000 patients undergoing one of six high-risk surgeries at nearly 2,200 hospitals across the country in 2005 and 2006. They then compared those patients’ outcomes to the hospitals’ track records on Hospital Compare. She said the researchers were surprised by the findings.

“We were expecting [the measures] to be closely tied to patient outcomes, and were instead surprised to find that hospitals [with high compliance rates] don’t have correspondingly lower rates of mortality, or complications,” she says.

The study ties into a larger discussion on how best to measure hospital quality. “Process” measures like those the in the study are one possibility — measuring hospitals based on how well they follow standard procedures designed to minimize risk and improve patient outcomes, like administering the right antibiotics and at the right times.

Hospitals can also be compared directly on the outcomes — in other words, how many patients survive after a particular surgery, or how many develop infections. But measuring hospitals based on outcomes data is complicated, because not all hospitals serve the same patients.

“It’s a tough line to walk, because it’s very difficult to adequately adjust for risk,” explains Rachel Werner, a health economist at the University of Pennsylvania who has studied hospital quality measurement. It might be that hospitals with high mortality rates are doing a bad job, but “It might be they’re just treating very sick patients. The worst case scenario is that a hospital just stops accepting sick patients [in order to get a better rating].”

Werner says the research in the area is evolving, and she believes it is moving toward using both process and outcome measures together, along with other “structural” measures, such as the number of nurses per patient at a particular hospital: “Obviously they can be used in combination, which I think is where most of these report cards are headed,” she says.

The discussion over measuring hospital quality is not new, but it is about to become even more important than it has been in the past, because under the health reform law hospitals’ pay will soon be tied, at least in part, to their performance.

Most hospitals have been required to report some quality measures to Medicare in order to receive their full annual payment update since the Medicare Modernization act of 2003. But beginning in October 2012, the health reform law will actually begin to tie hospitals’ Medicare reimbursements to how well they do on quality measures — though the details of what those measures will be is still being worked out.

Princeton health economist Uwe Reinhardt, who delved into the issues surrounding how to measure hospital quality in two recent columns on the New York Times’ Economix blog, says that despite the difficulties of accurately measuring hospital quality, he believes the focus on quality is worthwhile:

“You have a standards board that tries to refine it and get it better and better,” he says, “And gradually we do get better.”