The use of computers to replace tasks previously done by hand continues to become more prevalent, from using machine learning to analyse database studies, to algorithms that recommend whether someone should receive a bank loan or be shortlisted for a job interview. Another area that uses such predictive algorithms is in the criminal justice system, where they are often used to predict criminal behaviour, such as locations of crime ‘hotspots’, the likelihood of whether defendants will attend their court hearing, and/or whether someone will reoffend. However, there is concern as to the accuracy and fairness of these systems.
In an article in Science Advances, Dressel and Faris compared a commercially available criminal risk assessment tool against assessment by untrained participants on accuracy of deciding whether a defendant would reoffend within two years. These participants were recruited via an online system and paid $1, with a bonus of $5 if the accuracy of their predictions was high (to incentivise them to treat the task seriously). The computer algorithm assessed 137 features of 1000 defendants and their past criminal record, while the volunteers were given a statement containing seven features (sex, age, criminal history) of a subset of 50 defendants. Comparing the results showed no significant difference (p=0.045) between the accuracy of the algorithm (65.2%) and the participants (62.8%). Pooling the participant responses (‘wisdom of the crowd’) showed similar accuracy (67.0%) (p=0.85). Further analysis showed that participants’ prediction accuracy were slightly more sensitive and less biased than that of the algorithm; while they were similar in terms of fairness regarding race of the defendant. Perhaps with participants who are well versed in criminal justice, or who are well trained, their accuracy could be higher than that of the computer?
The authors then went on to recreate the accuracy of the commercial computer algorithm using a simpler standard linear predictor, and found that inputting only two features (age and total number of previous convictions) gave results as accurate as the algorithm using 137 features.
— Peter Chilton, Research Fellow
- Lilford RJ. Machine Learning and the Demise of the Standard Clinical Trial! NIHR CLAHRC West Midlands News Blog. 10 November 2017.
- Lilford RJ. Machine Learning. NIHR CLAHRC West Midlands News Blog. 11 November 2016.
- Dressel J & Farid H. The accuracy, fairness, and limits of predicting recidivism. Sci Adv. 2018; 4(1): eeao5580.
Big discontinuities are fascinating. Just when we think we understand something, the trend line changes radically. Examples of unexpected discontinuities in trends include the massive decline in smoking among African-Americans in the 1980s ; the drop in crime in high-income cities over the last decade or so ; and the recent drop in teenage pregnancy rates. These are favourable trends in contrast to the sudden end of year on year decline in mortality among the majority population in one large country – white people in the US. Anne Case and Angus Deaton drill down into the numbers in their recent paper:
- Is this trend confined to white people? Yes, black and Hispanic people continue to experience declining mortality rates.
- Is this trend seen in other high-income countries? No – in France, Sweden, Japan and the UK, age-specific mortality continues to decline across the populations.
- How does it differ among whites by economic class? Using education as a proxy, a decline in life expectancy is confined to those with no education beyond high-school.
- What diseases are driving it? ‘Deaths of despair’ (suicide, alcoholic cirrhosis, drug overdose) are rising among white people in the US in absolute terms, and in comparison with non-white groups and with other countries. Cardiovascular deaths are no longer declining among whites in the US, even as they continue to do so in other countries. Increases in ‘deaths of despair’ along with arrest in declining cardiovascular diseases, combine to extinguish the declining trend.
- Is the phenomenon localised geographically? No, the ‘epidemic’ in ‘deaths of despair’ among white people covers rural and urban areas, and has pretty much become country-wide.
- Is the problem gender specific? No, the rise in ‘deaths of despair’ among the less-educated group affects both women and men.
- What are the long term trends? While the differences in mortality between better and less well educated groups are getting narrower in Europe, the gap is getting wider among whites in the US. This widening gap is also reflected in changes in self-assessed health.
So is all this really just a reflection of widening economic disparities? No:
- Disparities are widening within the black community and between black people and white people. However, mortality is converging between rich and poor black and Hispanic people, and ‘deaths of despair’ are not increasing in these ethnic groups.
- Widening disparities are seen in all comparator countries – in Spain, ‘deaths of despair’ actually declined through a vicious economic downturn between 2007 and 2011, for example.
- The difference in outcome correlates much more strongly with change in education than change in income.
- Historically there are many instances when mortality and inequality have moved in different directions, and selective reporting can be used by unscrupulous ideologues to buttress either side of this argument.
So why has it happened. Here we need to turn to sociology (in some desperation). A novel, called ‘Fishtown’ (by Neal Goldstein) captures some of the sociology; a tale of a rising feeling of purposelessness as workers overseas and machines at home combine to force less educated people (men especially) out of jobs. Such people rely on welfare, while immigrants take over the lowest paid jobs. Another explanation turns on the idea of differentials – this time between whites and non-whites, and loss of status rather than failure to achieve it – “if you have always been privileged, equality begins to look like oppression.” Case and Deaton are careful to point out that the above explanations are not strongly supported by the data. But there is something ‘out there’ – a ‘latent variable’ with a long memory (i.e. operating over the life course of various ‘cohorts’ of people). Many commentators pretend they have understood these latent variables, but I think we are going to have to look a lot harder and resist the beguiling but facile explanations offered up by journalists, political commentators, and academics alike (a point pursued in the next exciting instalment of your News Blog).— Richard Lilford, CLAHRC WM Director
- Oredein T & Foulds J. Causes of the Decline in Cigarette Smoking Among African American Youths From the 1970s to the 1990s. Am J Public Health. 2011; 101(10): e4-14.
- The Economist. Falling crime. Where have all the burglars gone? The Economist. 20 July 2013.
- Wellings K, Palmer MJ, Geary RS, et al. Changes in Conceptions in Women Younger Than 18 Years and the Circumstances of Young Mothers in England in 2000-12: an Observational Study. Lancet. 2016; 388: 586-95.
- Case A, & Deaton A. Mortality and morbidity in the 21st century. Brookings Papers on Economic Activity. BPEA Conference Drafts. March 23-24, 2017.