This interesting paper  shows that risk-adjusted Patient Reported Outcome Measures (PROMs) produce more outliers by surgeon than Standardised Mortality Ratios (SMRs). Presumably the distribution of PROMs was more skewed than the distribution of SMRs. However, surgeons accounted for only a small proportion of all unexplained variance in the distribution of PROMs, so little could be achieved by weeding out a few individual surgeons. In addition, risk adjustment can under-adjust so that a true positive becomes a false negative. For example, controlling for hospital type could obscure poor performance in types of hospital that attract poorly performing surgeons. A better way to scrutinise surgical performance might be to analyse surgical skill by observing recordings of operations and analysing them by experts or automated systems. These results could be correlated with PROMs to identify truly excellent surgeons who could give proctorship to those whose processes and outcomes were less good. We think the future of quality monitoring lies in triangulating ‘performance data’ and using it in local improvement programmes.  — Richard Lilford, CLAHRC WM Director References:
- Varagunam, M; Hutchings, A, Black, N. Do patient-reported outcomes offer a more sensitive method for comparing the outcomes of consultants than mortality? A multilevel analysis of routine data. BMJ Qual Saf. 2015; 24(3): 195-202.
- Hampton T. Efforts seek to develop systematic ways to objectively assess surgeons’ skills. JAMA. 2015; 313(8): 782-4.
- Department of Health. Guidance on the routine collection of Patient Reported Outcome Measures (PROMs). London: Department of Health. 2009. [Online].
- Lilford R, & Rosser D. What’s Up Doc? Health Serv J. 2013; 123:19-21.