Resources to accompanying training on ‘Cognitive bias’

Late at night, at the end of a long shift, whilst patrolling for a suspect in a house robbery case, Police Officer X encounters person Z and shoots him dead.

After the incident the X reports that he believed Z had a gun, and that X was in imminent danger. A citizens group reports that there is a racist pattern in police shootings, such that officers are more likely to shoot at blacks than whites.

Z was black and unarmed.

X's decision was a mistake. How do you stop mistakes like this from happening in the future?

Notes for participants

This scenario is based on research into the so ­called ‘Shooter bias’ phenomenon (Correll et al., 2002, Mekawi and Bresin, 2015). Experiments show that UK and US participants are quicker to decide to shoot (in a simulation) a target if he is black compared to white. They are also more likely to mistakenly shoot an unarmed target if he is black compared to white.

The effect is also found in serving US police officers (Correll et al., 2007). Related work suggests that the results are not solely due to the thresholds people adopt for their decisions, but may reflect an increased likelihood of misperception,­ ie when the target is black, participants are more likely to actually ‘see’ a gun which isn't there.

One caveat to bear in mind is that although differential rates of shooting between black/whites indicate racial injustice, they aren’t certainly because of explicit or implicit bias on the part of individual police officers.

This New York Times article argues that structural factors, such as poverty, have a more powerful influence on who is involved in police killings than racial bias of individual officers:

Killings of blacks: Here is what the data say

BBC article: Black people 'three times more likely' to be tasered

The police officer’s dilemma - Pages from the researchers who elucidated this effect experimentally.

Correll, J., Park, B., Judd, C. M. and Wittenbrink, B. (2002) The police officer's dilemma: Using ethnicity to disambiguate potentially threatening individuals. Journal of Personality and Social Psychology, 83, 1314­1329.

Correll, J., Park, B., Judd, C. M., Wittenbrink, B., Sadler, M. S. and Keesee, T. (2007) Across the thin blue line: Police officers and racial bias in the decision to shoot. Journal of Personality and Social Psychology, 92, 1006­1023.

Mekawi, Y. and Bresin, K. (2015) Is the evidence from racial bias shooting task studies a smoking gun? Results from a meta-a­nalysis. Journal of Experimental Social Psychology.
View article

You are reviewing the hiring decisions in a large firm, after a complaint from an unsuccessful candidate for a team leader job.

The unsuccessful candidate has plenty of hands ­on experience of the work but fewer qualifications. The successful candidate had less hands ­on experience, but was well qualified.

The chair of the hiring committee justified the appointment by highlighting the importance of formal qualifications for the position.

The unsuccessful candidate alleges gender bias. She points to a parallel position which was filled in the firm, and for which an experience but under­qualified candidate was appointed over an inexperienced but well qualified candidate.

What advice would you give to chairs of future hiring committees so they could ensure a gender impartial hiring process?

Notes for participants

This scenario is based on experiments which showed that in a hiring choice for a new chief of police between a streetwise but uneducated and an educated but not streetwise candidates, participants shifted their criteria to emphasise the importance of the quality possessed by the male candidate. Their judgements of candidate quality were not effected by gender ­so they rated the CVs shown to them as equally high quality, but the suitability for the job was affected by gender. (Uhlmann and Cohen, 2005).

Also relevant is work which shows that the same behaviours can be interpreted differently depending on how they align with stereotypes or expectations. Brescoll and Uhlmann (2008) showed that a man shown reacting angrily was judged to be justified, his anger caused by something about the situation. A video a woman reacting angrily was judged to show her to be emotional, and her anger unjustified.

Brescoll, V. L. and Uhlmann, E. L. (2008) Can an angry woman get ahead? Status conferral, gender, and expression of emotion in the workplace. Psychological Science, 19, 268–275.

Uhlmann, E. L., and Cohen, G. L. (2005) Constructed criteria redefining merit to justify discrimination. Psychological Science, 16(6), 474­480.

It is the week before the start of a trial for a case which has gained local notoriety.

An editorial in the local paper, without mentioning any specifics of this case, sets out that crime has become ‘wild beast preying on the city’, and that it ‘lurks in every neighbourhood’.

On the first day of the case a crowd gathers outside the court, loudly demanding a 10-year prison sentence for the defendant.

How is the prosecution and sentencing of identical crimes influenced by the social and political context within which they happen?

Notes for participants

The editorial in the paper may shift what people think is a reasonable response to crime. Thibodeau and Boroditsky (2011) conducted an experiment where participants read one of two versions of a news story, that differed by the metaphor used to describe crime. The critical

differences, shown in {red}, were very small):

“Crime is a {wild beast preying on/virus infecting} the city of Addison. The crime rate in the once peaceful city has steadily increased over the past three years. In fact, these days it seems that crime is {lurking in/plaguing} every neighborhood. In 2004, 46,177 crimes were reported compared to more than 55,000 reported in 2007. The rise in violent crime is particularly alarming. In 2004, there were 330 murders in the city, in 2007, there were over 500.”

After reading the news story participants were asked about the appropriate policy responses to crime. Participants who read the piece with the beast metaphor were more likely to recommend responses focussed around punishment, such as longer custodial sentences or increased police presence.

Participants who read the piece with the virus metaphor were more likely to recommend responses focussed around prevention, eg economic regeneration, rehabilitation of offenders.

All of social psychology suggests that the opinion of other people has a pervasive and profound impact on our own opinions, so it would be bizarre if the crowd outside the courtroom didn’t influence those inside in some ways.

One additional influence may be the (legally uninformed) recommendation of a 10-year sentence. Research into the ‘anchoring effect’ shows that when thinking about numbers, including prices, probabilities or prison sentences, people are prone to incorporate irrelevant information into a basis figure (anchor) from which they adjust their judgement.

For example, people estimated Mahatma Gandhi’s age at death to be 50 if first asked the seemingly trivial question of whether he died before or after the age of 9. Those who were first asked if he died before or after the age of 140 estimated his age at death to be 67 - a 17 year difference. The idea being that the two numbers, 9 and 140, although incorrect, act as anchors from which people make their estimates of the true value.

There is evidence that judges aren’t immune to this effect. Enough and Mussweiler (2001) gave trial judges from one regional superior court (Landgericht) in Germany rape case materials containing either two-month or 36-month recommended sentences (the anchors). The difference in average recommended sentences, after the judges had reviewed the materials was 28 vs 36 months.

Experiments with mock juries (eg Chapman and Bornstein, 1996) showed that jurors can be influenced by anchors in compensation pay outs by hundreds of thousands of dollars.

Thibodeau, P. H. and Boroditsky, L. (2011) Metaphors we think with: The role of metaphor in reasoning. PLoS ONE 6(2): e16782. doi:10.1371/journal.pone.0016782

Enough, B. and Mussweiler, T. (2001) Sentencing under uncertainty: Anchoring effects in the courtroom. Journal of Applied Social Psychology, 31(7), 1535-1551.

Chapman, G. B. and Bornstein, B. H. (1996) The more you ask for, the more you get: Anchoring in personal injury verdicts. Applied Cognitive Psychology, 10(6), 519-540.

An accessible introduction to the anchoring effect

On decision making

These four books are all accessible guides to the psychology of decision making:

1. Reckoning with risk: Learning to live with uncertainty by Gerd Gigerenzer (2002)

2. The power of intuition: How to use your gut feelings to make better decisions at work by Gary Klein (2004)

3. The invisible gorilla: And other ways our intuitions deceive us by Christopher Chabris and Daniel Simons (2010)

4. Thinking, fast and slow by Daniel Kahneman (2011)

On bias

These are resources which might help understand how bias, sometimes subtle, can affect our decision:

• Reviewing applicants: Research on bias and assumptions, women in Science and Engineering Leadership Institute, University of Wisconsin-Madison (PDF, 189KB)

• Project Implicit (Harvard)

Resources for legal professionals

These are specifically for people working in courts or with the law:

• Parliamentary Office of Science and Technology (POST) note on ‘Unintentional bias in court’, October 2015

• National Center for State Courts, ‘Helping courts address implicit bias’

De-biasing

These scholarly papers discuss the evidence on improving decisions:

• Wilson, T. D. and Brekke, N. (1994) Mental contamination and mental correction: Unwanted influences on judgments and evaluations, Psychological Bulletin, 116(1), 1171-142.
  Classic review of type of bias

• Paluck, E. L. and Green, D. P. (2009) Prejudice reduction: What works? A review and assessment of research and practice, Annual Review of Psychology, 60, 339-367.
  Review of explicit prejudice and what works to reduce it.

• Moss-Racusin, C. A., van der Toorn, J., Dovidio, J. F., Brescoll, V. L., Graham, M. J. and Handelsman, J. (2014) Scientific diversity interventions, Science, 343(6171), 615-616.
  Recent review on evidence about effective diversity interventions.

• Milkman, K. L., Chugh, D. and Bazerman, M. H. (2009) How can decision making be improved? Perspectives on Psychological Science, 4, 379-383.
  Evidence of how decisions can be improved

• Dixon, J., Levine, M., Reicher, S. and Durrheim, K. (2012) Beyond prejudice: Are negative evaluations the problem and is getting us to like one another more the solution? Behavioral and Brain Sciences, 35(06), 411-425.
  Argues against focussing on individuals.

• Jolls, C. and Sunstein, C. R. (2006) Debiasing through law. The Journal of Legal Studies, 35(1), 199-242.
  Describes how legal procedure can interact with our psychological tendencies.

• Kassin, S. M. and Wrightsman, L. S. (1988) The American jury on trial: Psychological Perspectives. Taylor & Francis.
  Argues that legal system poorly accounts for some cognitive biases.

• Hahn, U. and Harris, A. J. (2014) What does it mean to be biased: Motivated reasoning and rationality. Psychology of Learning and Motivation, 61, 41-102.
  Argues that evidence of human ‘irrationality’ is mostly exaggeration.

• Croskerry, P., Singhal, G. and Mamede, S. Cognitive debiasing 1: Origins of bias and theory of debiasing. BMJ Qual Saf 2013, 22 (Suppl 2):58-64.

• Croskerry, P., Singhal, G. and Mamede, S. Cognitive debiasing 2: Impediments to and strategies for change. BMJ Qual Saf 2013, 22 (Suppl 2):65-72.
  Review from a medical decision making context.

Our work

Our project, funded by the Leverhulme Trust, has focussed on the analysing what implicit bias means for our ideas of responsibility for our (biased actions).

• Stafford, T. (2014) The perspectival shift: How experiments on unconscious processing don’t justify the claims made for them, Frontiers in Psychology, 5, 1067.
  View article

• Holroyd, J., Scaife, R. and Stafford T. (forthcoming 2017) What is Implicit Bias? Philosophy Compass.

• Holroyd, J., Scaife, R. and Stafford, T. (forthcoming 2017) Responsibility for

• Implicit Bias, Philosophy Compass.

Scholarly work on bias in courts

These are references in case you want to read the scholarly literature on bias in law:

• Greenwald, A. G. and Krieger, L. H. (2006) Implicit bias: Scientific foundations, California Law Review, 94(4), 945-967.

• Kassin, S. M., Dror, I. E. and Kukucka, J. (2013) The forensic confirmation bias: Problems, perspectives, and proposed solutions. Journal of Applied Research in Memory and Cognition, 2(1), 42-52.
  View article

• Casey, P. M., Warren, R. K., Cheesman, F. L., II and Elek, J. K. (2012), Helping courts address implicit bias: Resources for education, Williamsburg, VA: National Center for State Courts.

• Kang, J., Bennett, M. W., Carbado, D. W., Casey, P., Dasgupta, N., Faigman, D. L., ... and Mnookin, J. (2012) Implicit bias in the courtroom, UCLA Law Review, 59(5).
  View article

• Jolls, C. and Sunstein, C. R. (2006) The law of implicit bias, California Law Review, 94, 969-996. doi:10.2307/20439057

• Babcock, L., Loewenstein, G. and Issacharoff, S. (1997) Creating convergence: Debiasing biased litigants, Law and Social Inquiry, 22(4), 913-925.

• Farnsworth, W. (2003) Legal regulation of self-serving bias, The. UC Davis L. Rev., 37, 567.

• Koehler, J. J. and Meixner, J. B. (2013) Decision making and the law: Truth barriers. Wiley-Blackwell Handbook of Judgment and Decision Making, forthcoming, 13-04.