The Problem with ‘Red, Amber, Green’

I have always thought that the so-called traffic light system, which classifies service quality as ‘red, amber, green’ is so crude as to be otiose. A recent article in BMJ Quality and Safety explicates the logic behind my intuition.[1] There are two problems with the so-called traffic light system. First, it focuses just on one episode in time and says nothing about trends. Second, it (usually) relies on thresholds set externally. For both of these reasons, the authors argue (and I agree) that the traffic light system is inimical to safety. It does not distinguish between common cause variation (play of chance) and special cause variation (likely to be due to some specific, potentially remediable factor). The point is made that tackling common cause variation, even if it comes up red against some externally set threshold, is likely to lead nowhere. If one wants to improve outcomes from systems that are fluctuating randomly, then it is necessary to look for a common cause, not a cause specific to a particular time and space. Control charts analyse trends and hence distinguish between common cause variation and special course variation; the latter requires a focussed approach. For instance most English Accident and Emergency departments would show up red if judged against the four hour waiting time target. A red rating does not therefore suggest a problem specific to a particular hospital, but failure across the hospital system. It therefore needs a systemic approach across hospitals. This would be self-evident if a funnel plot were used. Such a chart would distinguish outliers where a targeted diagnosis and intervention would be appropriate from the generality of hospitals where a more systematic approach is more likely to bear fruit. CLAHRC WM is trying to enhance uptake of control charts by hospitals based on our previous work that shows they are seldom used.[2]

— Richard Lilford, CLAHRC WM Director

References:

  1. Anhøj J, Hellesøe A-MB. The problem with red, amber, green: the need to avoid distraction by random variation in organisational performance measures. BMJ Qual Saf. 2017; 26: 81-4.
  2. Schmidtke KA, Poots AJ, Carpio J, Vlaev I, Kandala N-B, Lilford RJ. Considering chance in quality and safety performance measures: an analysis of performance reports by boards in English NHS trusts. BMJ Qual Saf. 2017; 26: 61-9.
Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s