Data collection and interpretation are critical pieces in the forward progress of the U.S. healthcare system, especially as performance-based payment models become more popular. According to an American College of Physicians position paper, the use of performance measurement (PM) is “intended to help achieve improved quality, high-value care, better patient satisfaction, improved health outcomes, and lower costs.”
To leverage the full power of data, however, we need to go well beyond the basic measures of patient satisfaction or an overview of a physician group’s performance. Instead, we must assess physician care on an individual provider level using clear metrics and hard data. This was demonstrated in a 2019 study in The Journal of Bone and Joint Surgery, which showed how simply sharing a surgeon scorecard was associated with improved value in elective primary hip and knee arthroplasty. After nine months of scorecard distribution amongst orthopedic surgeons, the mean total costs for total joint arthroplasties decreased by 8.7%, with the mean total direct variable costs decreasing by 17.1% and implant costs decreasing by 5.3%. Length of stay also decreased slightly with no significant changes to readmission rates. Collecting and sharing individual physician metrics can have immediate positive effects.
While quality monitoring is becoming a more accepted method for providers, payers, and patients, some providers are slow to adopt evidence-based results because they argue that their personal perception of their patient’s outcomes does not allow them to easily transition to shared clinical pathways of care. Furthermore, there are several additional challenges currently inhibiting progress in improving and assessing performance, such as the following.
One of the biggest hurdles in measuring the quality of individual physician care is the difficulty of constructing valid measures. Obstacles include small sample sizes, non-comprehensive data sources, and non-standardized information systems—all of which lead to unclear outcomes. PM is especially divisive amongst physicians who look after high-risk patients due to unintended adverse consequences. However, a healthcare system can overcome these challenges by carefully creating its own standard data set to ensure comprehensive, relevant, and fair measures are used across all specialties and individuals. One critical component of data collection and analysis is implementing patient risk stratification so that more complex patients will not be avoided by providers.
The political issues associated with data collection are even trickier. One of the main things holding back PM is the wariness of physicians. While physicians are accustomed to being held accountable by their patients, many are uncomfortable with the idea of receiving a report card and being scrutinized and measured directly against their peers. Inevitably, physicians at the middle or lower level of quality care will be resistant to the process, even though in an ideal environment the results should help raise the overall level of quality for all involved. There are also rightful hesitancies over how data will be interpreted (or misinterpreted) by the public. Overcoming these complex hurdles lies in involving physicians throughout the entire process (more on this below).
In the world of data collection, the expression “garbage in, garbage out” holds true. Indeed, measuring the right data—in the right way—is the only way a quality measurement plan will be successful. Relevant metrics may include evidence-based measures of outcomes, patient experience, safety, effectiveness, population health, and total cost of care. However, before implementing this reporting, there are several methodological issues tied to physician-level data collection that must be resolved first.
Attribution rules determine which physicians are accountable for the care provided. The complexity here arises if a patient has seen multiple providers. For example, if a patient has consulted with a neurologist, an orthopedic surgeon, and a spine surgeon for their back injury, under whose name should the patient’s outcome be attributed? If an outcome is good, multiple physicians may want attribution. Conversely, if it’s poor, no one wants to put their name and reputation on it. Attribution rules must therefore be designed so that decisions are not made on a case-by-case basis for the sake of fairness and transparency. Here are a few types to consider:
It can be difficult to produce adequate sample sizes for individual physicians as they may not have sufficiently adequate numbers of patients with certain diseases or conditions to properly measure. Lower volumes in and of themselves are a concern for quality assessment as higher volumes usually improve consistency of care and reduce adverse outcomes. A 2008 study examining the reliability of quality measures to assess physician performance found that “reliability was low even when physicians had 50 quality events.” Therefore, it's important to put in minimum thresholds for data collection before reporting. However, in areas such as unique surgical procedures, 50 or fewer patients may be adequate to develop a strong patient care and quality assessment program.
Composite scores tabulate results across individual measures to create scorecards for a broad topic. Examining a combination of results for a broad topic can be a smart way to address small sample sizes. This might entail a particular aspect of care (e.g. surgical recovery) or a condition (e.g. osteoarthritis). Composite scores for surgical recovery might include opioid use, hospital readmissions, wound care, infection rates, significant complications, patient’s time to return-to-work, functional activities, and sports.
What’s more, a composite score system can compare surgeons to a single standard score, rather than directly comparing surgeons to one another within a department. According to a 2020 article in Advances in Orthopaedics, a composite score “may help facilitate surgeon growth without fostering unnecessary competition amongst colleagues.” Paradoxically, it is the spirit of competition that often drives overall quality to the highest level.
For the process to be fair, once results are collected, aggregated, and analyzed, they must also be validated. This could be done by having physicians review their own data to confirm the accuracy or by hiring third-party auditors to ensure a lack of bias.
Of note, physicians may be more willing to support the measurement program if they are involved in finalizing the results before they are submitted or made public, therefore allowing them to contest any that they feel are unjust. The only caveat, however, is that this approach can significantly slow down the process, so it may need to be adjusted or minimized as the program matures.
How To Implement Data Collection—And Get Physician Buy-In
As mentioned, the biggest challenge to measuring individual physician performance is physicians themselves. One of the low hanging fruits in cost containment is identifying any physicians within a healthcare system who have a clear pattern of ordering excessive imaging, shunning evidence-based best practices, or repeatedly performing unnecessary procedures. To identify physicians who are exhibiting this type of behavior, look for high variabilities with similar physicians with established benchmarks. While typically only a handful of physicians fall into this category, the resistance to PM can be widespread among many physicians. Any plan to implement data collection therefore needs to address this potential resistance from the beginning.
To begin with, physician voices must be heard when initially designing quality metrics and reporting processes. One of the best ways to accomplish this is by appointing physician leaders to work with administration to help manage the project and carry out the behavioral and performance changes that need to be made. Physicians will be most effective at communicating with other physicians and helping push forward change. Therefore, a dyad leadership model—comprising both administrators and physicians—is critical to the success of any data collection project.
Secondly, the actual collection process must be painless. The last thing physicians need is more barriers to productivity. Fortunately, sophisticated software platforms can be used to measure the quality of physician care. Changes to EMR workflows can be used to ensure physicians are not tasked with being the sole gatherers of data. Healthcare technology has become increasingly savvy and, as such, should be used as much as possible to track, measure, and analyze data. On the horizon are advances in machine learning and artificial intelligence that will further facilitate the collection and analysis of data. Therefore, it is critical physicians get involved now to ensure the most appropriate outcomes are being measured.
As the U.S. healthcare system continues to evolve at a rapid clip toward a truly value-based system, quality monitoring of physician behavior and results of care is essential to maintaining the appropriate checks and balances required to ensure patient experience and outcomes don’t suffer. It’s time to embrace the swaths of data at our fingertips and leverage its true power.