racial bias

machine learning in the criminal justice system


Criminal InJustice

Across the country, courts and probation or parole officers use classification algorithms to predict criminal defendants' risk of recidivism.1 Recidivism is the tendency of a released criminal to reoffend and end up back in prison. Many people, including myself, believe that the efficacy of a criminal justice apparatus should be measured by how well it reduces recidivism.

At all stages of the criminal justice system, defendants or convicted criminals are assigned a risk assessment score. The scores are used during bail or bond hearings. They are used when determining a parolee's supervision level. In nine states, risk assessment score are given to judges at sentencing.2 Presumably, defendants with a higher risk score would be given longer sentences.

ProPublica reviewed one of these algorithms - the COMPAS algorithm made by Northpointe, Inc. - and found that it performs too poorly to be used as it is currently used. They compared the scores assigned by COMPAS to over 10,000 defendants in Broward County, Florida, to the defendants actual future behavior. The algorithm correctly predicted recidivism only sixty percent of the time and correctly predicted violent recidivism only twenty percent of the time.

The algorithm has similar overall success rates regardless of defendants' race. But its mistakes differed depending on a defendant's race. The algorithm was twice as likely to misclassify black defendants as higher risk than they later turned out to be, and it was twice as likely to misclassify white defendants as lower risk than they later turned out to be.

The COMPAS tool is a sad example of how a model built on biased data will make biased predictions. I am not going to delve into how institutional racism manifests in the criminal justice system. Instead, I will focus on how classification algorithms are optimized and tested, and when they should and shouldn't be used.

Designing a classification model

Inspired by ProPublica's review of the COMPAS algorithm, I wanted to design my own algorithm for predicting recidivism. First, I wanted to see if I could make an algorithm with any improved performance. A sixty percent accuracy rate is rather poor. Second, even while purposely excluding race as a model input, I wanted to see if my model exhibited any racial bias.

To start, there are two main considerations when designing a classification algorithm: what types of errors to avoid and how interpretable the model should be.

Minimizing errors

Below is the confusion matrix for predicting recidivism. A false positive is incorrectly predicting that a person will reoffend. This COMPAS algorithm has a higher false positive rate for black defendants than for white defendants. A false negative is incorrectly predicting a person will stay out of prison. The COMPAS algorithm has a higher false negative rate for white defendants than for black defendants.

Model Predictions
Reoffend Don't Reoffend
Defendants' Actual Behavior Reoffend True Positive False Negative
Don't Reoffend False Positive True Negative

The criminal justice system is founded upon the principal that as a society we would rather let a guilty man walk free than send an innocent man to prison. This is the why we presume all defendants are "innocent until proven guilty." Accordingly, I opted to minimize the model's false positives at the expense of allowing more false negatives. A false positive could send a low risk person to prison for longer than is warranted. For this model, I view a false negative as giving a defendant the benefit of the doubt.

The COMPAS algorithm was only sixty percent accurate, but accuracy is the wrong metric for this situation. I will optimize my model on precision. Precision is a ratio of true positives to all positive predictions. A model that is 100% precise would never say an innocent man is guilty, but may mislabel some guilty men as innocent.


Ignoring computation speed, predictive models are a tradeoff between performance and interpretability. A black-box model could be correct every time, but you wouldn't be able to look under the hood and see what leads to its classifications. Any steps you take to make the model more transparent degrade its results.

Sometimes you don't care about interpretability. If I was designing a vision system for a self-driving car, as long as it always saw the person in the crosswalk, I wouldn't care how it works. But if I am going to stand in a court room and say a defendant will likely reoffend, that defendant will probably want to know why I say so. Simple saying, "the computer says so," won't cut it.