Patterico's Pontifications

2/27/2010

Statistics Expert: L.A. Times “Mischaracterized” Key Statistic in Front-Page DNA Article

Filed under: Crime,Dog Trainer,General — Patterico @ 2:09 pm

Statistics expert Prof. David Kaye today writes that the Los Angeles Times mischaracterized a crucial statistic in a front-page 2008 article about DNA and statistics.

Regular readers know that the L.A. Times‘s series of flawed articles on DNA, cold cases, and statistics is a longstanding complaint of mine. In May 2008, The Times ran an article about the case of John Puckett that claimed that “the probability that the database search had hit upon an innocent person” was 1 in 3. This phrasing told readers that, based on the relevant DNA statistics, there is a 1 in 3 chance Puckett was innocent. But the statistics did not support this claim.

The issue comes up again now because the Washington Monthly this week published an article repeating the false suggestion:

[John] Puckett was arrested, tried, and eventually convicted based mostly on the DNA match, which was portrayed as proof positive of his guilt—the jury was told that the chance that a random person’s DNA would match that found at the crime scene was one in 1.1 million.

If Puckett’s were an ordinary criminal case, this figure might have been accurate. . . . But when suspects are found by combing through large databases, the odds are exponentially higher. In Puckett’s case the actual chance of a false match is a staggering one in three . . .

. . . Jurors told the Los Angeles Times that the one-in-1.1-million statistic had been pivotal to their decision. Asked whether the jury might have reached a different conclusion if they had been presented with the one-in-three figure, juror Joe Deluca replied, “Of course it would have changed things. It would have changed a lot of things.”

Prof. Kaye succinctly explains that this is misleading:

What did the Los Angeles reporters who interviewed the poor juror say that “1 in 3″ meant? Their article mischaracterizes it as “the probability that the database search had hit upon an innocent person.” As noted above, no one knows the probability that this search hit upon an innocent person. We know only that if Puckett and everyone in the database were innocent, then the chance that at least one person could have matched could have been no larger than about 1/3.

Concerning the Washington Monthly article, Prof. Kaye told me:

A definite tilt toward one side, I’d say. If investigative journalists presented a balanced and nuanced story, would they have a cause celebre? The author is right about one thing — people do not understand statistics. Indeed, the author himself perpetuates the silliness of comparing the Arizona database size to the random match probability for a specific nine-locus match that came from an all-pairs trawl of the database, fails to mention the strongly opposing view in the statistical community about the relevance of the Np statistic as an indication of the false positive probability, etc.

These errors, of course, originated in a series of similarly flawed articles in the Los Angeles Times by Maura Dolan and Jason Felch, of which the article on the Puckett case was only one part. The links debunking Dolan and Felch’s numerous errors are collected here.

But the worst error was the “1 in 3″ claim made in the Puckett aticle. This error was never corrected. It should be. The error has been repeated endlessly, in Law and Order TV shows, by Ed Humes, and most recently in the Washington Monthly article. People have been led to believe by the Los Angeles Times that there is a 1 in 3 chance that Puckett is innocent. As Prof. Kaye explains, that is a conclusion that is not supported by the available evidence and statistics.

6 Responses to “Statistics Expert: L.A. Times “Mischaracterized” Key Statistic in Front-Page DNA Article”

  1. On the average most people major in journalism or English because they can not hack the math required in more academic fields. Or not.

    Huey (efe02b)

  2. As someone who had no trouble with 3 semesters of calculus and minimal trouble with differential equations, I eagerly look to someone else to explain statistics to me. Once you get beyond 50-50 for a coin toss, things seem to get complicated amazingly quickly in my opinion.

    In addition, there is the profound statement on this subject reportedly from Mark Twain (there are 3 kinds of lies; lies, damned lies, and statistics).

    I agree with Prof. Kaye, most reporters don’t know the difference between the size of the locus and the size of a locust, (neither do most lawyers or doctors, but that is beside the point) and “all-pairs” has nothing to do with raising children.

    And just what is the significance of the Np statistic?
    (Pons, Blair, others who may know, you don’t need to answer).

    MD in Philly (e347b2)

  3. By the way, for about 2 seconds (literally) I referred to Prof. Kaye as a “DNA expert” in the title and once in the post. I immediately corrected those to “statistics expert” once I saw the published version, since that’s really what he is (though he writes about DNA in the context of statistics).

    Patterico (c218bd)

  4. After my attempt at humor dealing with just the statistical math, I will bring up another point for serious consideration.

    It shouldn’t take a PhD in logic to figure this out:

    Joe was convicted in 1986 of a brutal rape. Eighteen years later a sample from the crime scene is tested for DNA and is found not to match Joe, but someone else, who is in prison for other sexual assault crimes. The press reports this as “…served 18 years in prison for a 1985 sexual assault he did not commit“.

    To be precise, on the DNA evidence alone nothing has been proven whether he had a role in the original crime or not. All they had was the negation of one piece of evidence.

    It was ignored that prior to his arrest for that crime he had a record of increasing criminality, including dowsing a live cat with gasoline and lighting it on fire. It was also ignored that he wrote letters to his (ex-)wife and children that he was going to “get her” when he got out of prison. And it was ignored that he spoke to other inmates of plans to kidnap, terrorize, rape, and kill when he had the opportunity, describing in detail the handcuffs and leg irons he would use to restrain his victim.

    All of those things were no longer ignored after “Joe”, otherwise known as Steven Avery, was arrested and later found guilty for a brutal rape-torture-suicide 2 years after he was released. In that crime he involved another person, used the restraints he previously described, and burned the remains of the victim in a bonfire.

    I first heard of this some time ago when he was first arrested on suspicion of the proven crime. At the time, it was reported that the MO of this crime had many similarities to the attack he reportedly “did not commit”, but i can’t find that info now. It would be important to examine the exact findings and interpretation of the data rather than rely on a brief summary, but that is an investigation I can’t pursue

    I’m all for ways of exonerating the innocent, including DNA testing. But I hope that conclusions of innocence are “jumped to” any more than conclusions of guilt.

    http://www.jsonline.com/news/wisconsin/29388834.html

    http://www.powells.com/biblio?show=HARDCOVER:USED:9780743299367:18.95&page=excerpt

    MD in Philly (e347b2)

  5. > I’m all for ways of exonerating the innocent, including DNA testing. But I hope that conclusions of innocence are “jumped to” any more than conclusions of guilt.

    The presumption of innocence rule applies, though. If you’re going to “jump” to something, it should be innocence.

    I suggest that, if the remaining evidence does not preponderously suggest guilt, at the very least the accused deserves a new trial based on the remaining evidence. It is a miscegenation of justice to do otherwise. In general, I’d suggest that, if new evidence which significantly suggests innocence DOES come to light, then anyone convicted should almost automatically, if they request it, get a new trial.

    IgotBupkis (79d71d)

  6. IgotBupkis,

    I agree with you. What I don’t agree with is the idea that when DNA evidence is shown to be faulty it appears it is assumed the person is innocent without examining the rest of the case. I think the burden of proof is still on the prosecution to say that there was enough evidence other than the DNA not to scrap the whole case. I also think a new trial at least is in order to see if the conviction would hold up in light of the conflicting DNA evidence. I readily admit there may be many cases where a new trial is not even needed, as the rest of the case is weak in light of the conflicting DNA, especially if the DNA is linked to a specific suspect that fits the crime.

    I am eager to see innocent people go free. I think we can agree that the 1 in a hundred that should still be locked up remains so.

    MD in Philly (e347b2)


Powered by WordPress.

Page loaded in: 0.5067 secs.