Some people have tried to tell the CLAHRC WM Director that Statistical Process Control (SPC) and standard statistics are completely different ideas – one to do with special cause variation and the other with hypothesis testing. This concept is not so much wrong as it is not right! Fretheim and Tomic recently published a relevant and interesting article in BMJ Quality and Safety. The CLARHC WM Director’s take on this article is as follows. If there is no intervention, then use SPC to detect non-random variation, but if there is an intervention point (or period), use a statistical test. Since such a test uses the information that an intervention has occurred at a certain point in time, it is much more sensitive to change than SPC. Interrupted time series methods should be used to compare the slope of the lines before and after the intervention and remember to allow for any auto-correlation, as emphasised in a previous post. However, the CLAHRC WM Director emphasises the need for contemporaneous controls whenever possible to allow for temporal trends – ‘rising tide’ situations. He is also very concerned about publication bias arising from selective reporting of ‘positive’ interrupted time series studies. In the meantime, our CLAHRC has discovered that information presented to hospital boards mostly does not use SPC and when it is used, the limits (e.g. two or three SDs) are not stated.
— Richard Lilford, CLAHRC WM Director
- Fretheim A, & Tomic O. Statistical Process Control and Interrupted Time Series: A Golden Opportunity for Impact Evaluation in Quality Improvement. BMJ Qual Saf. 2015. [ePub].
- Chen Y, Hemming K, Stevens AJ, Lilford RJ. Secular trends and evaluation of complex interventions: the rising tide phenomenon. BMJ Qual Saf. 2015. [ePub].
- Schmidtke K, Poots AJ, Carpio J, et al. Considering Chance in Quality and Safety Performance Measures. BMJ Qual Saf. 2016. [In Press].