Discontinuities can be very revealing in Service Delivery and Policy Research – they provide a statistical method to detect the distorting effects of incentives. For example, the statistical test for p-hacking reported previously in your News Blog, is based around the p<0.05 threshold for statistical significance. While the p-value is easily ‘hacked’ by selectively reporting ‘significant’ results, other data may be harder – death rates for example.
The great American economist Raymond Fisman (he of the New York traffic violations fame) and Yongxiang Wang examined industrial deaths in China. A threshold for such deaths was set at national level, with a penalty for Provincial administrations who failed to reach the target threshold. The distribution of deaths across provinces looked like this before the incentives went live:
After the incentive, it looked like this:
Not only that, but this discontinuity is found exclusively in reports from the fourth quarter of the year. This makes a compelling case – if you provide a target and managers do not think it is fair, then they will manipulate it, even if it is something that, on the face of it, is hard to manipulate. You would not succumb to such a temptation, do I hear you say? But you would, oh yes, you would!
— Richard Lilford, CLAHRC WM Director
- Lilford RJ. Look out for ‘p-hacking’. NIHR CLAHRC West Midlands News Blog. 11 September 2015.
- Fisman R, Miguel E. Cultures of Corruption: Evidence from Diplomatic Parking Tickets. NBER Working Paper No. 12312. 2006.
- Fisman R & Wang Y. The Distortionary Effects of Incentives in Government: Evidence from China’s “Death Ceiling” Program. Am Econ J Appl Econ. 2017; 9(2): 202-18.