Some people have tried to tell the CLAHRC WM Director that Statistical Process Control (SPC) and standard statistics are completely different ideas – one to do with special cause variation and the other with hypothesis testing. This concept is not so much wrong as it is not right! Fretheim and Tomic recently published a relevant and interesting article in BMJ Quality and Safety.[1] The CLARHC WM Director’s take on this article is as follows. If there is no intervention, then use SPC to detect non-random variation, but if there is an intervention point (or period), use a statistical test. Since such a test uses the information that an intervention has occurred at a certain point in time, it is much more sensitive to change than SPC. Interrupted time series methods should be used to compare the slope of the lines before and after the intervention and remember to allow for any auto-correlation, as emphasised in a previous post. However, the CLAHRC WM Director emphasises the need for contemporaneous controls whenever possible to allow for temporal trends – ‘rising tide’ situations.[2] He is also very concerned about publication bias arising from selective reporting of ‘positive’ interrupted time series studies. In the meantime, our CLAHRC has discovered that information presented to hospital boards mostly does not use SPC and when it is used, the limits (e.g. two or three SDs) are not stated.[3]
— Richard Lilford, CLAHRC WM Director
- Fretheim A, & Tomic O. Statistical Process Control and Interrupted Time Series: A Golden Opportunity for Impact Evaluation in Quality Improvement. BMJ Qual Saf. 2015. [ePub].
- Chen Y, Hemming K, Stevens AJ, Lilford RJ. Secular trends and evaluation of complex interventions: the rising tide phenomenon. BMJ Qual Saf. 2015. [ePub].
- Schmidtke K, Poots AJ, Carpio J, et al. Considering Chance in Quality and Safety Performance Measures. BMJ Qual Saf. 2016. [In Press].
NIHR CLAHRC West Midlands News Blog 
I think using a hypothesis test for any intervention assumes that the intervention is 100% applied from that point. This is not always the case, particularly with a behaviour change intervention (e.g. hand washing) – althought could work for a situation where a drug is completed replaced for another.
Separately, interrupted time series are senstive to outying data (as all with all regression) and spuriously large effects may be found asa result of data at teh break point and end point.
Finally, looking at the data set presented in the paper, it seems rather conveninetly constructed to show SPC as a straw-man: a gentle curvi-linear relationship with lots of noise, Easy to put two straight line through, but with consderable inherent variation in which special cause is not strong.
I suspect, as with most applicaitons of statistics, care is required in selecting one’s analysis, which will depend on the purpose.
Whoever was talking to the Director didn’t express it quite right. It’s not about the difference between special causes and hypothesis testing. Both techniques are hypothesis testing, it’s just that they are addressing two very different questions. Both these branches of statistics were developed precisely because these questions require a different approach. I’m not able to access the article quoted (because I’m not an academic) but I can share another article published in the same journal by a respected statistician that makes the point much more eloquently than I can. The article is “Analytical studies: a framework for quality improvement design and analysis” by Lloyd Provost and published in qualitysafety.bmj.com on March 31, 2011