Yet Another Flawed Study on Crime
A piece in Slate is titled The Irrational 18-Year-Old Criminal. Its subtitle is: “Evidence that prison doesn’t deter crime.”
That headline should have you chuckling. But you’ll chuckle even harder if you read the article and understand the pseudo-science forming the evidentiary basis for this ridiculous conclusion.
If the only point of the article were to discuss the possible effect of increased penalties on criminals, that would be one thing. But the study described in the piece, as it is described, doesn’t even come close to supporting the broader claim that “prison doesn’t deter crime” as to society at large.
Details in the extended entry.
The article says:
Both the prospect of getting caught and the prospect of spending time in prison are supposed to deter forward-looking, rational potential offenders from criminal activity, encouraging more-constructive pursuits like staying in school or at least making French fries. More mechanically, prison also prevents crime by simply caging dangerous people. Deterrence has long been an article of faith among economic theorists and, more recently, economists who do empirical work, too. But now a series of careful studies by economists at Columbia and the University of Michigan are calling into question whether either policing or punishment successfully deters crime.
(My emphasis.)
The methodology is laughably flawed, because it doesn’t just look at “potential offenders” who might be deterred, but rather only at actual offenders who haven’t been deterred:
David S. Lee of Columbia and Justin McCrary of Michigan . . . noted that when kids turn 18, they suddenly face much stiffer adult sanctions. Then they got access to data on all felony arrests in Florida between 1989 and 2002. Each arrest links to an individual, whose birth date is included in the data. This allowed the researchers to create an arrest history for each person arrested and to measure the effect of turning 18, and thus facing longer prison terms, on criminal activity.
Note clearly: the pool of people in the study includes only people who have been arrested at least once in their life. Looking at these people only, the economists found that the possibility of greater punishment at age 18 had no measurable effect:
In Florida during the years in question, Lee and McCrary found, the probability of being sentenced to prison for an offense jumped from 3 percent to 17 percent at exactly age 18. This tees up the answer to the economists’ main question: How does the tendency to commit crimes vary around the 18th birthday, when the odds of a prison-sentence punishment jump? The answer is, hardly at all. While the probability of being arrested each week falls steadily from age 17 to age 19, there is no sizeable decrease in the arrest rate that corresponds to the bump up to an adult penalty in the weeks before and after people turn 18.
By restricting the subjects to people who have been arrested at least once, the economists skew the results — apparently without even realizing it. Unlike people who have never been arrested, people who have already been arrested have already proven that they are less likely than the average person to be deterred by the prospect of incarceration to begin with.
If these economists really wanted to look at how the threat of incarceration affects deterrence, they should look at the whole population. Instead, they focus precisely on the pool of people least likely to be deterred — a pool that has already proven that, in the past, the threat of incarceration did not deter them from behavior that caused them to be arrested.
By analogy, if you wanted to find out whether flu shots work, you would establish a control group of people who did not get flu shots, and compare it to a group that did, to see who ended up getting the flu. The goal would be to determine what effect flu shots would have on the entire population of people who got a flu shot. The approach described in the article, by contrast, would be like restricting your experiment to the subset of people who had a flu shot, but got the flu anyway — and then giving them two flu shots, to see if administering more flu shots will work better. The logic goes like this: if two flu shots don’t work any better than one, then, by God, flu shots must not work at all!
This “logic” would be flawed because it would completely ignore the people for whom flu shots do work.
Similarly, this study ignores the very people who are most likely to be deterred by the possibility of incarceration: people who are, in fact, deterred to begin with — to the point that they have never gone and gotten themselves arrested. Those people don’t figure into the study in any way that I can tell.
Restricting the study to people who commit crimes — or at least get arrested for them — magnifies the distortion caused by another problem with the study. This problem is a common one among studies by economists: the assumption, made by so many economists, that the players involved have perfect information. This is such a common character flaw among economists that it is the basis for many jokes, one of which goes something like this:
An old joke has an economist and a trader walking down the street. The trader says to the economist: “There’s a $20 bill lying on the sidewalk.” The economist replies: “No there’s not.” The trader responds: “But it’s right there!” “No,” the economist patiently explains, “if there was a $20 bill lying there someone would have picked it up already.” At this point, the trader leans down, picks up the $20 bill and they continue on their way.
The point is simple: economists measure human behavior by assuming perfect information on the part of actors. This is a generally useful assumption for measuring economic factors. However, it can become problematic if you forget that it’s not realistic. It can be especially problematic if you’re examining a group consisting of people who are less likely than average people to have good information — or to use it when they have it.
As a general rule, that is a good description of people who get arrested for crimes, which is why it is dangerous to extrapolate their behavior to the entire population.
The criminal defendants I see tend to have a poor understanding of the potential consequences of their behavior, at the time they engage in the behavior. What’s worse, even when they do have this information, they often ignore it. If there is one characteristic that is widely shared by people who face criminal charges in the courthouse where I work, it is that they tend to have very poor judgment. That’s what caused them to commit crimes to begin with. It’s what caused them to get caught. And often, it’s what causes them to take actions that are clearly not in their own best interests.
It is always a risky thing for scientists to assume perfect information is held by any group of people. But it is especially risky for them to assume that perfect information is held by people who get arrested for crimes. And it is foolish to look at the decisions made by those people, and to extrapolate them to society at large, and to assume that people who have never been arrested would react in exactly the same way to the possibility of incarceration.
Yet that’s exactly what these economists appear to be doing, based on the description provided in the article.
The problem gets worse when you’re looking at juvenile criminals who are turning into adult criminals. Anyone familiar with the juvenile criminal justice system knows that, whatever its benefits might be, it is not a system that teaches young criminals that crime has consequences. Arguably, it has the opposite effect on many a young offender. A typical 14-year-old caught stealing a car or breaking into a business is going to be sent “home on probation” with instructions to obey Mom and Dad and get good grades. You have to work really hard to get yourself confined at the next level of publishment, which is literally called “camp.” Only the very worst of the worst get incarcerated at the California Youth Authority. These are the kids who commit the violent crimes, often involving firearms or serious violence, and/or kids who are so undisciplined that they can’t function at home or in “camp.”
The juvenile justice system is so lenient in California that, if you are 13 years old, you can murder as many people as you like, as brutally as you like, with absolute premeditation — and you cannot by law be incarcerated past your 25th birthday. Even if you are a serial rapist-murderer with 30 victims, all of whom were horribly tortured, the state must let you go at age 25.
When one of these young thugs turns 18, they often have a short lifetime of crime under their belts already — and distressingly often, it’s a lifetime that has resulted in no consequence that a rational person would say fits the crime.
Now when they turn 18, if they commit a stick-em-up robbery with a gun — something that might have gotten them “camp” at one point in their life — they’re looking at 12 years in prison at 85%. And that’s if they plead guilty. It may be more if they go to trial.
Did these kids know they would be facing this kind of time? Did they have “perfect information”? In a lot of cases, I think they didn’t. And even if they did — even if someone once told them that things would be different at age 18 — they weigh that abstract knowledge against a lifetime of having received minimal punishment for their crimes. In that context, they may pay no attention to their abstract knowledge that consequences get worse when you’re an adult.
Does it make sense to look at the behavior of people with that sort of background, and extrapolate it to society at large, to conclude that prison does not deter criminal behavior by the average citizen? Of course not. Your average citizen is much closer to having perfect information than these new adults who just finished a juvenile life of crime. Your average person knows that armed robbery will result in a stiff prison sentence. These young criminals often don’t. They’ve had a lifetime of being taught that the punishment is never really that bad. I bet a lot of them are shocked to learn that they’re really facing 12 years for an armed robbery.
Now, once they’ve actually served their first adult sentence, they may be closer to perfect information. Maybe the study should look only at second-time adult offenders who have already experienced adult consequences for crimes committed as an adult. But it would be very hard to take important variables into account in a study. You’d have to control for a disposition to commit serious crimes — a disposition that may be unrelated to the question of whether they will face significant time in custody. After all, someone who commits an armed robbery may well have other problems in their life that outweigh their ability to rationally weigh the consequences.
But the fact that this is true of someone who has already committed one or more crimes doesn’t mean that it’s true of everybody. Plenty of people might be on the fence morally about whether to commit certain crimes, and may decide that the prospect of incarceration is enough to deter them. Some of those people might be minimally deterred by lesser punishments and more so by greater punishments. But you can’t draw these conclusions only by looking at people who have been arrested. Those are the people who aren’t getting deterred anyway.
If the point of the article is that criminals tend not be be deterred by greater punishment, there may be some minimal value to the study — though, like almost all criminal justice studies, the controls are almost certainly inadequate to reach even that modest conclusion.
But to say that the study provides “[e]vidence that prison doesn’t deter crime”?
Give me a break.
P.S. Sorry this post was so long. As the saying goes, I didn’t have time to make it shorter.
Reminds me of the “studies” in the 1950s that linked the reading of comic books with juvenile delinquency. The researchers discovered that nearly all juvenile delinquents had at one time or another read comics books, therefore comic books caused delinquency!
Perfect Sense (b6ec8c) — 1/31/2007 @ 2:48 amI’m trying to figure something out. If a group of college freshmen can understand, on a nearly-instinctual level, that asymmetric information is almost necessarily guaranteed when not directly prohibited (even without the benefit of their ECON 1100 course, which drills it in even better), why do practicing economists forget? Using a pure model approach for projecting human behavior seems to neglect the “human” aspect of things. In a different analogy (along the lines of your flu shot analogy, but one specifically economics-related), it would be as if a form was distributed with every box of (for example) Raisin Bran that asked how large of a concern price was when choosing whether or not to buy the cereal, then claiming that “consumers don’t care about price” if the results show a low concern. By only surveying those that have already bought the cereal, you’ve gotten skewed data.
Maybe we need to demand that all studies from now on involve a five-year-old with an overdeveloped sense of playground fairness to critique their methodology.
Rick Wilcox (71646f) — 1/31/2007 @ 5:11 amNow when they turn 18, if they commit a stick-em-up robbery with a gun something that might have gotten them camp at one point in their life theyre looking at 12 years in prison at 85%.
What does that mean — “at 85%”? They have an 85% chance of being sentenced to 12 years? They are likely to serve 85% of a 12 year sentence? Or something else?
[They’ll serve 85 percent of the time. Sorry, I was tired when I finished the post and didn’t edit it well. Hence the length. I’ll update and clarify. — P]
aunursa (be8887) — 1/31/2007 @ 6:04 amWe know that prison does deter crime in some countries so the solution would seem to be to follow the example of those countries.
Libs love to tell us about how crime-free Saddam’s Iraq was. It might benefit us to learn how Saddam’s prisons deterred crime.
J Curtis (d21251) — 1/31/2007 @ 6:29 amThere’s another flaw in the study: it ignores the findings of other (presumably better designed) studies showing that adolescents in general have poorer judgment. Which means that what a typical 18 year old might do (whether or not he/she has an arrest record) is probably more irrational than the general population to begin with.
So the study only shows that people who are expected to act more irrationally, actually act more irrationally.
kishnevi (6273ad) — 1/31/2007 @ 6:51 amIt seems likely that the prospect of incarceration deters some people but not others. Therefore, the net effect is one of deterrence.
dchamil (5ea287) — 1/31/2007 @ 7:08 amRidiculous conclusion from a study that is not remotely designed to test that question.
Some empirical evidence here. I have had numerous witnesses in my career as a prosecutor who have stated that they “did not want to get involved” in whatever crime by one of their friends because they were strikers or didn’t want to go to prison. I have also had a few witnesses/defendants who did “get involved” but clearly out of a moment of weakness (poor impulse control). I know from other aspects of those cases that they did not participate in all of their co-defendant’s crimes because they were worried about the penalty. I would like the study (or the article) to account for this empirical evidence.
David (fda1c0) — 1/31/2007 @ 7:31 amDear Gentlemen: I would be grateful if anyone could tell me what the rank of Cfn. stands for.. in the Canadian Infantry Corps tfd. R.C.E.M.E. in WWII. Thank-you.
Anastasia (fe7ba9) — 1/31/2007 @ 8:09 am.
But that’s just it: there really is no rational calculation by which any crime could be justified. Everybody knows that the clerk in a Seven-Eleven can’t get to more than $30 or $40, and that if you walk in and flash a gun, bam! that’s five years, minimum, in most places. Yet how many idiots walk in, wave a gun, knowing that they can’t get more than $40 and a twelve-pack? There’s just no rational calculation under which this makes sense, even if you have only a one in a thousand chance of getting caught.
Dana (3e4784) — 1/31/2007 @ 8:15 amA serious question, though I wouldn’t expect anyone to ever answer positively: is there anyone that anyone here hates badly enough that he would murder that person if he knew he’d get caught, but that the penalty was only one year in prison?
Dana (3e4784) — 1/31/2007 @ 8:18 amHmm, did you read the actual study? If not, you should probably hold off on criticizing the authors themselves — there’s a long, rich tradition of journalists reading scientific studies and jumping to simplistic but “sexy” conclusions that the scientists themselves didn’t suggest (or even specifically cautioned against). Just from the paper’s abstract, I get the sense that the authors didn’t draw such far-reaching conclusions, and that they address the possibility that the population in question is perhaps not aware of the increased jeopardy after age 18 (assuming that’s what they mean by “myopic”).
kenB (1ad56f) — 1/31/2007 @ 8:55 amthere’s a long, rich tradition of journalists reading scientific studies and jumping to simplistic but “sexy” conclusions that the scientists themselves didn’t suggest (or even specifically cautioned against).
On the flip side, there is also a strong tradition of putting sexed-up conclusions in the abstracts of studies and then contradicting this conclusion further inside. The various studies put out by the CDC in the 90s about gun violence are classic examples.
Darkmage (c20107) — 1/31/2007 @ 10:26 amDana, #10, I will answer in the affirmative. I think. I don’t know exactly how I would react if things were that way, and sitting here i could easily justify murder if the penalty was low enough. a few years ago, i almost committed a serious crime after a major (negative) life event, and the only thing that did deter me was the probability of not seeing my kids for 25 to life. had the max sentence been that low, and in my diminished mental and emotional state, i quite possibly would have gone through with it. I can’t guarantee that, and I don’t feel that way any more, but then? It’s a real possibility.
Pol Mordreth (e154a8) — 1/31/2007 @ 10:50 amI read this site daily, but I rarely comment, so here goes.
I think it’d be better to read the actual study, as opposed to relying solely on a slate summary. Here is a link to the pdf: http://www-personal.umich.edu/~jmccrary/crime_myopia.pdf
From my cursory reading of the paper, it seems that the authors are considerably more careful about their conclusions. They do, in fact, discuss the problems with assuming perfect information, and they realize that the study is far from perfect. It should be noted, though, that it’s only possible to study the data that is available. It would be exceedingly difficult to study how well prison sentencing deters crime among people who aren’t criminals.
In the policy implications section of the paper, they say that their results would suggest that as far as deterrence is concerned, it would be better to increase the likelihood of being caught as opposed to increasing the length of incarceration. That’s in an ideal world, but I’d be interested to know what people think of that conclusion.
Adam (40d1a3) — 1/31/2007 @ 10:55 amI love the blog that you have. I was wondering if you would link my blog to yours and in return I would do the same for your blog. If you want to, my site name is American Legends and the URL is:
http://www.americanlegends.blogspot.com
If you want to do this just go to my blog and in one of the comments just write your blog name and the URL and I will add it to my site.
Thanks,
J. Mark English (15bc19) — 1/31/2007 @ 4:12 pmMark
Perhaps a variant of increasing the likelehood of getting caught would be to make the consequences of serious juvenile crimes more serious. If you teach serious offendors that nothing really bad will happen for five years or so, then is it any suprise that they continue in the same behavior pattern once they are 18?
Garrett (c7e5d0) — 1/31/2007 @ 5:11 pmWithout impugning on someone’s liberties, it would be difficult if not impossible to ‘increase the likelihood of getting caught.’
The authors should have used every adult that hasn’t been to prison as the ‘deterred’ group. Yes not everyone hasn’t committed a crime because of deterrents (some people just never feel the need/have the reason) but if you are going to use the people that have done time as one group, the other group would have to be the ones that havn’t.
Lord Nazh (3465cc) — 1/31/2007 @ 5:20 pmLordNazh:
I don’t think the point of the paper was that prison is not a deterrent. It was that increasing sentence length in our current system does not have a strong deterrent effect. In other words, if, for instance, a thief received 5 years prison time for his crime, he wouldn’t be significantly deterred if the sentence were 7 or 10 years instead. However, if there were more police patrolling the streets, he might be deterred by the increased likelihood of arrest.
I think another interesting example is illegal downloading of copyrighted work. Many people download, knowing that it is illegal, but that the likelihood of getting caught is very low. This is despite the fact that once caught, the penalties can be quite severe (thousands of dollars in fines for stealing an album that would cost only a few dollars). I’ll bet that the number of people who pirate music would decrease dramatically if the likelihood of being was caught was increased to, say, 1 in 10, even if the penalty was reduced to a small fine (a couple hundred dollars).
Obviously, there are multiple components to deterrence, including both penalties and the likelihood of arrest. Finding the right balance would be the goal in an ideal world. That’s what I got from the paper.
Adam (40d1a3) — 1/31/2007 @ 5:34 pmAll you prosecutors out there know that what is missing in the lives of the juveniles in the system (generally) is a strong, male figure who excerts some discipline, and has a sense of right and wrong. For the most part, we (society) are raising generations of feral children, and then wondering why the end up spending the majority of their years behind bars.
Another Drew (8018ee) — 1/31/2007 @ 6:22 pmI’m a juvenile public defender (and long time lurker) so here’s my $.02–
My big mistake in accepting a job as a juvie PD was thinking that I would be practicing law. I’m part big sister, part pissed off auntie and all social work.
I was shocked…and I do mean shocked…at the absolute crap job their parents (if they have any) are doing. There is no thought to consequence because at home and school there are NONE. Mom is out trying to find a new boyfriend and Dad is in jail/on meth/drunk/etc. And the school is fed up trying to figure out what to do with them.
At home, if there is discipline it comes only in the form of physical. That’s it–whup the kid and go back to the couch. In school, their hands are tied. The IEP that most of my kids have leave few options.
More than that, the vast vast majority of these kids have a diagnosis of ADHD, Bi-polar and a sleep disorder (which is why they are unable to get to school). Naturally, I have to explain to kid and Mom that repeatedly saying ‘Fuck you, Bitch’ to a teacher is not a recognized symptom of any of these disorders. Most do not take this news well.
It goes without saying that the parents take these diagnosis as complete absolution to be parents. Generally speaking, the parents think their duties are complete if they remember to give their kids their meds half the time.
All of that said, I love my job. These kids matter to me. And maybe I get to be one of the few people who tell them “That is NOT acceptable!” and give them a reason. If my only victory is to get the kid to say “Yes Sir” to the judge and not yawn/stretch when in front of the bench this time (no, I’m not kidding) then it’s a win.
No matter what, I get up there and do my best for for them. Sometimes they notice but mostly not. And every once in a while I get a kid who’s actually innocent and, most importantly, convince the judge of it.
How to fix the juvie system? No clue.
Sorry for the rant, vent and way too long post.
no one important (1a1cf4) — 1/31/2007 @ 7:16 pmNice to hear from a lurker. I’d say that you are indeed someone important — very important to these kids.
Where do you work?
Patterico (a8fa4a) — 1/31/2007 @ 7:57 pm@dana:
assistant devil's advocate (035fd1) — 1/31/2007 @ 7:59 pmin accordance with your proposed program, can i get a group rate of just four years in prison if i go out and murder five different people?
As a retired Probation Officer permit me to observe that swiftness and certainty of punishment is more important as a deterrent than severity of punishment. I don’t have any data to back this up, but if you are a parent you know it’s true.
#20 no one important? But you are.
FWIW when I had a probationer who was committing technical violations as opposed to criminal activity my last step before bringing him back to court for a violation hearing was to call his public defender, explain the situation and ask said defender to try and contact the probationer and urge him to co-operate with me. In every case they responded. It was a waste of time to ask retained counsel to do this.
Stu707 (5b299c) — 1/31/2007 @ 8:02 pmNo One Important,
You sound like a classy PD to me. Sometimes we do things that have good consequences we never know about, and my guess is that’s especially true in your case.
DRJ (e69ca7) — 1/31/2007 @ 8:26 pmI didn’t read the study, but the possible implication of this description of it is that the study concludes that policing might not deter crime.
[My live preview is not working, so it is with some trepidation that I submit this comment, except that any due punishment will not deter me, anyway, so don’t bother.]
J. Peden (ce006c) — 1/31/2007 @ 8:52 pmPeden:
I haven’t read the study carefully. But, that’s not what it said in the abstract or the parts of the text that I read. I could be wrong, though.
Adam (e13086) — 1/31/2007 @ 9:09 pmThanks for the welcome.
I’m in a semi-rural county in the Mid-South. We have a fairly active criminal docket given our size of around 45k.
So many of my kids are low functioning which, predictably, impacts their ability to forsee consequences. This begs the question of how to address the reasons for the higher rates of lower functioning individuals and what effect would that have on criminal activity?
Beware–bleeding heart, tree hugging is about to show–Continue at your own risk
What effect does infant/toddler stimulation, or lack there of, have? What about lack of proper nutrition (processed, preservative laden, chemicals rather than fresh fruits/veggies/un-treated meat)? What about the lack of daily role models, especially male as Another Drew pointed out? What about lack of EMPLOYED role models (disablity having replaced welfare) to demonstrate how to get and keep a job? What about non-incarceration, swift consequences (me–understandable to the low functioning) as Stu707 mentioned? What about lack of basic parenting skills?
Those are questions I would like answered when addressing how to deter criminals.
BTW, my post name is because I’m too lazy to think of something better.
no one important (1a1cf4) — 1/31/2007 @ 10:52 pmAdam:
If longer sentences aren’t a ‘significant’ deterent, then by virtue that makes prison not significant.
As I said: come up with how it’s more likely to get caught. More police on the streets does not correlate to more chances to get caught (unless you blanket police coverage, but with budgets and what not, that doesn’t seem likely).
I wasn’t talking so much on what the study implied (prison vs sentences) as to their methods. You have to use two groups to do a scientific study, one group is the prisoners then the other group would have to be the non-prisoners 🙂 would make an interesting study.
If you link every criminal to the fact that they weren’t deterred (nevermind the WHY of the actual crime) then you have to link every non-criminal to being deterred (again, disregarding why).
Lord Nazh (3465cc) — 2/1/2007 @ 8:08 am#27
Many of the above are functions of parental poverty. There are some 37 Million people living in poverty, but only a tiny fraction become delinquents/criminals. Social scientists should study people living in poverty in inner cities and rural areas who do not become criminals and try to find out why.
Stu707 (5b299c) — 2/1/2007 @ 10:50 amI agree with Patterico that the study is nonsense (not to use a stronger word). I also agree that most criminals are risk-takers whose judgment as to the potential rewards and punishments for their behavior is flawed. Ken Lay and Tom Skilling are perfect examples, in my opinion.
I think that deterrence is a red herring. Who the hell cares whether my neighbor deters me from beating my wife if he goes to jail for beating his wife? Stop the bastard from beating his wife!!! (JUST AS AN EXAMPLE, MY NEIGHBORS ARE VERY GOOD PEOPLE WHO LOVE THEIR WIVES.)
Incapacitation, stopping the criminal from commiting more crimes, is all the justification needed for prisons and the death penalty.
nk (5e5670) — 2/1/2007 @ 11:30 amP.S. Sorry. Jeff not Tom. Although the example still works. Tom, honest weatherman. Ken, thief.
nk (5e5670) — 2/1/2007 @ 11:39 amno one important,
I don’t think there are easy answers to your questions but let me suggest another aspect for your consideration: genetic causes. I am not suggesting that people are born criminals but I do think some people are born with a predisposition to more willingly take risks.
Risk-taking is not a bad quality. In fact, it comes in handy for people who work in the military or in police and fire departments. Most of us shirk from danger but there are (fortunately) people who, with training and equipment, are willing to take risks.
This may be why education is important and especially why mentoring works so well with some kids, who need help channeling their risk-taking proclivities into appropriate venues. It might also explain their willingness to engage in behavior that can have adverse results – such as early experimentation with “exciting” behavior like criminal or gang activity, alcohol and drug use, and teenage sex that results in unplanned pregnancy – that typically leads to fewer positive life choices down the road.
DRJ (e69ca7) — 2/1/2007 @ 12:11 pm