Science & the Public | Science News

MISSION CRITICAL

Support credible science journalism.

Subscribe to Science News today.

Science & the Public

Where science
and society meet

Science News

Science & the Public


Science & the Public

Are computers better than people at predicting who will commit another crime?

Maybe not

person in handcuffs

SO-SO SOFTWARE When it comes to predicting whether or not someone will commit another crime — which can affect lockup time — computer programs don’t have an edge over people.

Sponsor Message

In courtrooms around the United States, computer programs give testimony that helps decide who gets locked up and who walks free.

These algorithms are criminal recidivism predictors, which use personal information about defendants — like family and employment history — to assess that person’s likelihood of committing future crimes. Judges factor those risk ratings into verdicts on everything from bail to sentencing to parole.

Computers get a say in these life-changing decisions because their crime forecasts are supposedly less biased and more accurate than human guesswork.

But investigations into algorithms’ treatment of different demographics have revealed how machines perpetuate human prejudices. Now there’s reason to doubt whether crime-prediction algorithms can even boast superhuman accuracy.

Computer scientist Julia Dressel recently analyzed the prognostic powers of a widely used recidivism predictor called COMPAS. This software determines whether a defendant will commit a crime within the next two years based on six defendant features — although what features COMPAS uses and how it weighs various data points is a trade secret.

Dressel, who conducted the study while at Dartmouth College, recruited 400 online volunteers, who were presumed to have little or no criminal justice expertise. The researchers split their volunteers into groups of 20, and had each group read descriptions of 50 defendants. Using such information as sex, age and criminal history, the volunteers predicted which defendants would reoffend.

A comparison of the volunteers’ answers with COMPAS’ predictions for the same 1,000 defendants found that both were about 65 percent accurate. “We were like, ‘Holy crap, that’s amazing,’” says study coauthor Hany Farid, a computer scientist at Dartmouth. “You have this commercial software that’s been used for years in courts around the country — how is it that we just asked a bunch of people online and [the results] are the same?”

There’s nothing inherently wrong with an algorithm that only performs as well as its human counterparts. But this finding, reported online January 17 in Science Advances, should be a wake-up call to law enforcement personnel who might have “a disproportionate confidence in these algorithms,” Farid says.

“Imagine you’re a judge, and I tell you I have this highly secretive, highly proprietary, expensive software built on big data, and it says the person standing in front of you is high risk” for reoffending, he says. “The judge would be like, ‘Yeah, that sounds quite serious.’ But now imagine if I tell you, ‘Twenty people online said this person is high risk.’ I imagine you’d weigh that information a little bit differently.” Maybe these predictions deserve the same amount of consideration. 

Judges could get some better perspective on recidivism predictors’ performance if the Department of Justice or National Institute for Standards and Technology established a vetting process for new software, Farid says. Researchers could test computer programs against a large, diverse dataset of defendants and OK algorithms for courtroom use only if they get a passing grade for prediction.

Farid has his doubts that computers can show much improvement. He and Dressel built several simple and complex algorithms that used two to seven defendant features to predict recidivism. Like COMPAS, all their algorithms maxed out at about D-level accuracy. That makes Farid wonder whether trying to predict crime with anything approaching A+ accuracy is an exercise in futility.

“Maybe there will be huge breakthroughs in data analytics and machine learning over the next decade that [help us] do this with a high accuracy,” he says. But until then, humans may make better crime predictors than machines. After all, if a bunch of average Joe online recruits gave COMPAS a run for its money, criminal justice experts — like social workers, parole officers, judges or detectives — might just outperform the algorithm.

Even if computer programs aren’t used to predict recidivism, that doesn’t mean they can’t aid law enforcement, says Chelsea Barabas, a media researcher at MIT. Instead of creating algorithms that use historic crime data to predict who will reoffend, programmers could build algorithms that examine crime data to find trends that inform criminal justice research, Barabas and colleagues argue in a paper to be presented at the Conference on Fairness, Accountability and Transparency in New York City on February 23.

For instance, if a computer program studies crime statistics and discovers that certain features — like a person’s age or socioeconomic status — are highly related to repeated criminal activity, that could inspire new studies to see whether certain interventions, like therapy, help those at-risk groups. In this way, computer programs would do one better than just predict future crime. They could help prevent it.

Psychology,, Science & Society

You’ve probably been tricked by fake news and don’t know it

By Erika Engelhaupt 6:00am, December 4, 2016
In the fight against falsified facts, the human brain is both the weakest link and our only hope.
Technology,, Science & Society

Obama worried about research funding

By Janet Raloff 11:25am, April 30, 2013
Barack Obama offered yet another argument about why the current federal-budget stalemate is so risky: “[T]he sequester, as it’s known in Washington-speak — it’s hitting our scientific research.” As things now stand, “we could lose a year, two years of scientific research as a practical matter, because of misguided priorities here in this town.”
Technology,, Humans & Society,, Life & Evolution,, Genes & Cells,, Earth & Environment,, Ecology

Antarctic test of novel ice drill poised to begin

By Janet Raloff 12:37am, December 15, 2012
Any day now, a team of 40 scientists and support personnel expects to begin using a warm, high pressure jet of water to bore a 30 centimeter hole through 83 meters of ice. Once it breaks through to the sea below, they’ll have a few days to quickly sample life from water before the hole begins freezing up again. It's just a test. But if all goes well, in a few weeks the team will move 700 miles and bore an even deeper hole to sample for freshwater life that may have been living for eons outside even indirect contact with Earth’s atmosphere.
Humans & Society

This snowbird is really going SOUTH

By Janet Raloff 9:30am, December 6, 2012
Many people of a certain age (like my folks) enjoy flying south to warmer climes when winter weather threatens. I’m also flying south this December — but not to warm up. As a guest of the National Science Foundation, I’ll be checking out summer in the really deep South: Antarctica. Temps expected at certain sites I’m scheduled to visit, such as the South Pole, threaten to surpass the worst that my hometown will encounter in the dead of winter.
Animals,, Humans & Society,, Earth & Environment

Epidemic of skin lesions reported in reef fish

By Janet Raloff 5:58pm, August 1, 2012
A British-Australian research team has just found coral trout living on the south side of the Great Barrier Reef sporting dark skin raised, scablike, brown-black growths. Although the authors believe they’ve stumbled onto an epidemic of melanoma — a type of skin cancer — other experts have their doubts. Strong ones.
Humans & Society

So long Weekly Reader . . .

By Janet Raloff 12:12am, July 26, 2012
I read with sadness this week that Weekly Reader is about to disappear. As much as I’ll miss the idea of the venerable Weekly Reader living on, I also have to admit to a bit of a love/hate relationship with it. This conflict developed shortly after I joined the staff here. As soon as I identified my affiliation, people frequently asked: “Science News — hmmm: Isn’t that the Weekly Reader of science?”
Technology,, Humans & Society,, Nutrition,, Earth & Environment

FDA bans BPA in baby bottles, cups

By Janet Raloff 5:51pm, July 17, 2012
From now on, U.S. manufacturers may no longer produce polycarbonate baby bottles and sippy cups (for toddlers) if the clear plastic had been manufactured from bisphenol A, a hormone-mimicking compound. Long-awaited, the announcement is anything but a bold gesture. The Obama administration decided to lock this barn door after the cow had died.
Humans & Society,, Earth & Environment,, Chemistry,, Body & Brain

Putting BPA-based dental fillings in perspective

By Janet Raloff 6:29pm, July 16, 2012
A new study finds that children who have their cavities filled with a white composite resin known as bis-GMA appear to develop small but quantifiable drops in psychosocial function. To put it simply: Treated kids can become more moody, aggressive and generally less well adjusted.
Humans & Society,, Ecology

Warning to bats: Cuddle not

By Janet Raloff 4:57pm, July 5, 2012
Ecologist Kate Langwig of Boston University and her colleagues want Eastern bats to listen up: No more cuddling — at least during hibernation. Just keep those wings to yourselves.
Subscribe to RSS - Science & the Public