21st Jun 2012 posted in Education
Medicare is starting to reimburse physicians based in part on the “quality” of their care. To incentivize better results, the theory goes, doctors whose patients’ health improved could get reimbursed at a higher rate
Ah, but how to measure quality fairly? What about all the things that doctors can’t control? Patients who are obese, patients who don’t even bother to fill their prescriptions… Surely a doctor can’t be blamed if these patients fail to thrive.
Mathematica released a new paper yesterday on whether docs could be evaluated based in part on value-added models similar to the ones designed for teachers (pdf here). Doctors everywhere shuddered, no doubt.
Here’s the part that makes theoretical sense: Doctors, like teachers, have not historically been paid based on how much they actually help their charges. In fact, doctors are often rewarded to do things that may be directly counter to patients’ interests (like administering tests they don’t need). It would make sense if doctors got a small bump if, for example, their diabetic patients’ glucose levels improved over time. That is a hard thing to do—and it should be rewarded.
The Mathematic paper suggests that Medicare could, for example, compare a patient’s glucose levels from one year to the next, and see how the change compared to the change in levels of similar patients in the same region. After that, of course, things get complicated—just as they do with teachers.
What about patients who have many doctors? What about doctors who see the same patients for many years?
The answer might be the same as the proposed fix for teachers, the paper suggests: Multiple measures!
In education, approaches to increasing the precision of performance estimates include using test scores from multiple years of classes (in other words, increasing the sample size for the estimate); combining value-added scores with other, independent measures of teacher performance, such as principals’ evaluations; and calculating scores at a higher level of aggregation (e.g., for all the teachers in a given subject or for all the teachers in a school, which, again, increases sample size). The following similar approaches could be taken in health: using multiple years of patient’s outcomes; combining value-added measures with other measures of physician performance, such as their scores on clinical process measures; and calculating value- added scores for groups of physicians in a practice.
Soon the model becomes very complex; and if most doctors do not buy in, and I suspect they will not, many will find a way to game the system. They will resent the intrusion. Some may drop low-income, low-performing patients. Others may indeed respond to the incentives—for ego or financial reasons—and actually improve their practice to lead to better outcomes.
Who knows? Maybe the value-added model could work even better with doctors than teachers. Unlike teachers, many doctors get into the profession partly to make money; they may, theoretically (again!), be more motivated by pay-for-performance schemes than teachers. They also have enough schooling that many of them may actually understand the models—unlike most people (including most teachers). And there may be more visibility into what they need to do in order to get better outcomes than there is for teachers.
Or not. So far, there’s not a lot of evidence that using value-added models to evaluate teaching actually improves outcomes for kids in the real world. There’s not a lot of evidence that it doesn’t. Time may tell.
Regardless, thinking about this analogy made me realize that teachers have way more time to try to influence students than doctors have to affect patients. On the high end, let’s say a doctor sees a patient every two months for 15 minutes. Teachers, meanwhile, spend about 5 to 25 hours a week with their students. Over the course of two months, then, teachers have somewhere between 16,000% to 80,000% more time with students than doctors have with patients. (Granted, teachers don’t see their students one-on-one for all that time, but you get the idea: teachers spend a LOT of time with their students.) Opportunity lurks in that time….
Time is something that adults underestimate, I find. Perhaps we think it moves more swiftly because we have less of it left. Kids on the other hand know what it feels like to sit in the same room for 5 hours for 185 days. Much can happen in that time, they will tell you. (Or not.)