Talk:Prosecutor's fallacy

From Wikipedia, the free encyclopedia

You have new messages (last change).

This is the talk page for discussing improvements to the Prosecutor's fallacy article.

Contents

[edit] original article on the so called Prosecutor's fallacy

From the original article on the so called Prosecutor's fallacy:

Consider for instance the case of Sally Clark, who was accused in 1998 of having killed her first child at 11 weeks of age, then conceived another child and killed it at 8 weeks of age. The prosecution had an expert witness testify that the probability of two children dying from sudden infant death syndrome is about 1 in 73 million. To provide proper context for this number, the probability of a mother killing one child, conceiving another and killing that one too, should have been estimated and compared to the 1 in 73 million figure, but it wasn't. Ms. Clark was convicted in 1999, resulting in a press release by the Royal Statistical Society which pointed out the mistake (http://www.rss.org.uk/archive/reports/sclark.html).

The reason the case against Clark was flawed has nothing to do with the Prosecutor's fallacy, or the misapplication of statistics; though I agree that the way in which statistics were used here was simply wrong.

The case against Clark was flawed because it rests on a false dichotomy: Either Sally Clark's children both died from sudden infant death syndrome or she killed them.

While it's true that SIDS and infanticide are contrary explanations, showing one to be false is insufficient by itself to establish the truth of the alternative.

True, but nobody, not even the defense, introduced any other explanations; essentially everyone agreed that those two were the only possible explanations in that case. That should definitely be stated in the article though. AxelBoldt 20:28, 28 Jul 2004 (UTC)
One issue...SIDS and Infanticide still don't work together in a simple binomial distribution. They're not the only possibilities, despite the lack of any others being brought up. By a simple True/False logic, there is a 100% chance that either one of these possibilities is true. We see Cromwell's rule, that assuming the probability of 1 in a set of observations that could most definitely be expanded upon, we ignore all outside data not covered. Now, to get to the point, I quote a previous user; "While it's true that SIDS and infanticide are contrary explanations, showing one to be false is insufficient by itself to establish the truth of the alternative." That's true. If the prosecuter argued on this fallacy, his claim of a 1 in 73 million chance of the children dying of SIDS would naturally have to make the incredible assumption that there is a 72,999,999 in 73 million chance that the children were killed by her. That is, of course, if he stuck to the simple binomial idea of one idea being true, then the other must be false. I'm sure we can agree that that claim is absoutely ridiculous by a statistical standpoint. The original entry was correct in assuming that the probability of the murder should have been calculated and compared to this probability. As far as I can tell, the entry is correct as it stands.

Perhaps the assumption that these were the only two possibilities should be introduced into the article, as user AxelBoldt stated. Yes, it's a very likely assumption, and one that would be correct. I would see that as the reason that the statistics were misused; a simple misconception of the false assumption of a binomial distribution. It's said that statistics lie, and they can be warped very easily.

I removed:

Of course some consider the Prosecutor's fallacy no fallacy at all, because a logical fallacy, by Wikipedia's own definition, is any way in which an argument fails to be valid or sound. Based as it is upon probability, the Prosecutor's fallacy is not a deductive argument to begin with, and therefore questions of its validity or soundness are themselves variants of the fallacy of stolen concept.

I don't think anyone, not even prosecutors or Bayesians, consider the reasoning behind the Prosecutor's fallacy to be sound. Furthermore, reasoning about probabilities can very well be fallacious, like the Gambler's fallacy for example.

I also cut the remainder of the article significantly, pointing out that some Bayesians may find it justified to use an a priori probability of 1/2, which is what the Prosecutor's fallacy implicitly assumes. AxelBoldt 18:58 Oct 13, 2002 (UTC)

I don't see how the part of the article that says

One formulation of Bayes' theorem then states:
Odds(G|E) = Odds(G) · P(E|G)/P(E|~G)

is justified. Using Bayes' theorem to find the probablility of guilt gives:

Odds(G|E) = Odds(G) · P(E|G) / P(E)

where due to mutual exclusion:

P(E) = P(E|~G) · P(~G) + P(E|G) · P(G)

The article's formula holds only when you make the additional, unstated assumption that P(G) is small enough to make P(E|~G) a good approximation for P(E).

That's not incorrect; the formula Odds(G|E) = Odds(G) * P(E|G)/P(E|~G) is correct, without any unstated assumptions. To prove the formula, divide Pr(G|E)=Pr(E|G)*Pr(G)/Pr(E) by Pr(~G|E)=Pr(E|~G)*Pr(~G)/Pr(E). AxelBoldt 20:28, 28 Jul 2004 (UTC)

Does this happen everywhere int the World or limited to in U.S.? -- Taku

No, it has occurred in cases in the UK -- Tony Vignaux 09:00 18 May 2003 (UTC)

I'm not sure that the Clark example illustrates the Prosecutor's Fallacy very well, because I think there's another closely related fallacy. I think I understand the Bayesian analysis, but I think there's a perhaps more obvious error than not computing the prior Odds(G): the calculation of Odds(G|E) doesn't take into account that any family that has 2 children who die of SIDS could find themselves in the same situation as Clark, and there are probably millions of such families.

Suppose 50% of people are guilty and we believe that the probability of two children in the same household dying of SIDS are 1 in 73 million. We still shouldn't be convinced that Clark is guilty. Rather, if there are say 10 million households with two children, and any time two children die there's a police inquiry because it's suspicious when 2 children in the same household die, then the odds of an unfortunate 2-time SIDS mother coming before the court is approximately 1/7.

BTW, the fallacy of the other example given in this article seems more related to the number of people tested than to failing to take into account the a priori probability.

Here's a an example of not taking into account the a priori prob, that has no problem with the number of people tested: suppose I believe that I have magical abilities that allow me to win the lottery, even though the odds of winning with random numbers are 1 in 1 billion. I play and win the lottery. Are you convinced I have magical abilities? I'd say it's certainly worth checking out, but anyone's reasonable estimate of the a priori odds that I have magical powers is very very low. So, it's more likely that I was simply very lucky than that I have magical powers. Note that this example doesn't require any argument about how many other people declare that they have magical powers and play the lottery -- I could be the only one, and my claim would still be dubious.

What are people's thoughts on my reasoning here? If I'm sound, does anyone know which is the prosecutor's fallacy: failing to account for the a priori probability, or failing to account for the number of tests done?

Zashaw 05:21 24 May 2003 (UTC)

I don't exactly follow your alternative analysis (and I am sure there aren't "millions of families" with 2 SIDS deaths). Prosecutor's fallacy, to put it succinctly, is the assumption that tiny Pr(~G|E) yields similarly tiny Pr(~G). I think there are various possible reasons that can cause a person to commit this fallacy. AxelBoldt 20:28, 28 Jul 2004 (UTC)

In another scenario, assume a rape has been committed, and all the males of the town are rounded up for DNA testing. Finally one man whose DNA matches is arrested. At the trial, it is testified that the probability of finding a DNA match is only 1 in 10,000. This does not mean that the suspect is innocent with the tiny probability of 1 in 10,000. If for instance 20,000 men were tested, then we would expect to find two matches, and the suspect is innocent with probability at least 1 in 2.

This was confusing. If all the men in the town have been tested, and there's only 1 match, then that's your guy. So I replaced the above with a version that didn't say all the men had been tested. Does it look OK? Evercat 13:30 14 Jul 2003 (UTC)

It's unfortunate that people change text that they simply don't understand. The statement is that 1 in 10,000 people will match a given DNA sample. Given 20,000 people, 2 are likely to match -- they can't both be guilty. That means that DNA testing is not 100% accurate, thus if there's only one match, that does not prove that it's your guy.


I found the resulting paragraph a bit confusing, because at first it wasn't stated who was tested, other than the indicted man. So, I changed it to just say that 20,000 men were tested, & didn't say how this related to the size of the town. BTW, though, the example is valid even if everyone in the town is tested, since it's not clear that the rapist necessarily lived in that town -- he could easily have come from a neighboring town.
I also didn't see how the probability 1 in 2 was arrived at, at the end of the pragraph, so I did the calculation I think more accurately.
Zashaw 22:58 19 Jul 2003 (UTC)

[edit] spurious argument

I removed the following paragraph from this article: Another instance of the prosecutor's fallacy is sometimes encountered when discussing the origins of life: the probability of life arising at random out of the physical laws is estimated to be tiny, and this is presented as evidence for a creator, without regard for the possibility that the probability of such a creator could be even tinier. There is absolutely no way to establish the probability of there being a creator. One person might say the probability is zero. Another might say that it's 1. A few may put it somewhere in between. But the question isn't really one of probability at all. :-Rholton 02:40, 3 Dec 2004 (UTC)

That's not a valid reason to remove the paragraph. The described argument takes the probability of there being a creator as the inverse of the probability of life arising at random out of physical laws. Within that framework, the argument is an example of the prosecutor's fallacy. Whether that inverse relationship really holds or whether its actually possible to calculate the probability of life arising at random out of physical laws is beside the point. 68.6.73.60 11:50, 13 March 2006 (UTC)

[edit] Strange statement

This seems to be incorrect, unless I'm missing something:

When the photographic evidence is combined with the match, the two together point strongly towards guilt, since (assuming the chance of being in the photograph and having the match are independent) the chance that the accused is innocent falls to about 0.01%.

It seems that the probability of innocence only falls to 89.1%, since 0.9 x 0.99 = 0.891. Even with the untenable independence assumption (for example, someone may be framing them by setting up several pieces of evidence), it would take quite a bit more evidence to show anything beyond reasonable doubt. Deco 07:16, 17 Feb 2005 (UTC)

You are indeed missing something and falling into the defendant's fallacy. The argument goes that the prior probability that the man is innocent is 9,999,999/10,000,000. While the likelihood of having the match and being in the video may be 1 if guilty, the likelihood of the match if innocent is 1/1,000,000, and the likelihood of being in the video if innocent is 1/100,000, so (assuming independence since this is mathematics, not real life) the likelihood of both happening if innocent is 1/100,000,000,000. That gives a posterior probability of being innocent of 9,999,999/100,009,999,999 which is 0.000099989991... or about 0.01%. --Henrygb 03:11, 12 Mar 2005 (UTC)


Thanks, I get it now - I forgot we were dealing here with conditional probabilities, not just prior probabilities. This result fits with common sense, too — we often consider 2 pieces of strong evidence conclusive. Deco 03:30, 12 Mar 2005 (UTC)

Who originated the terms "prosecutor's fallacy" and "defense fallacy." Shouldn't there be a reference to the origin of these terms?


I'm sick and tired of the phrase "a concrete example".

[edit] Right answer to the wrong question

This is rather minor, doesn't take anything away from the article, but in the interests of accuracy...

At the top, the article mentions a 1-in-a-million chance in a community with 10 million people, saying that on average 10 people will match. Then later it says that this means that if someone matches it only means a 1/10 chance of guilt. However this is not the case - if we know someone matches it changes the expected number of people that will match. There is one person (the defendant) who we know matches, and there are 9,999,999 people who each also have a 1-in-a-million chance of matching, fro which we can expect about 10 people to match. This gives the defendant a 1/11 chance of guilt. Of course this figure changes again if numerous tests were performed and only the defendant matched - it's 1 in (1 + one millionth of the untested population).

Though I don't want to add this straight into the article, I want to get some confirmation I'm not completely wrong in my calculations here... most notably that I'm not falling on the wrong side of the trap "Expected number given this particular person matches" vs "Expected number given at least one person matches" - I think I have it right though. Phlip 06:49, 6 October 2005 (UTC)

[edit] A surprising example

A friend of mine sent me this example. It may deserve a paragraph here or in the Bayes theorem article. Bill Jefferys 21:36, 16 December 2005 (UTC)

[edit] Possibly erroneous statement

"When the photographic evidence is combined with the match, the two together point strongly towards guilt, since (assuming the chance of being in the photograph and having the match are independent) the chance that the accused is innocent falls to about 0.01%. This low probability of innocence is not proof of guilt." - the derivation of 0.01% is unclear. This could be wrong.

I said the same thing. See above. Perhaps this should be clarified in the article. Deco 18:52, 22 December 2005 (UTC)

[edit] Another possibly erroneous statement

In legal terms, the prosecutor is operating in terms of a presumption of guilt, something which is contrary to the normal presumption of innocence where a person is assumed to be innocent unless found guilty. A more reasonable value for the prior odds of guilt might be a value estimated from the overall frequency of the given crime in the general population.

In the UK at least, procecutors are oblidged to presume guilt, reguardless of evidence to the contrary.

Defendants are oblidged to presume innocence, reguardless of evidence to the contrary (unless their client pleads guilty).

jurors are oblidged to initially presume innocence, and change their mind according to the cases presented, but only return a guilty verdict if they are sure 'beyond reasonable doubt' of the defendant's guilt.

And, finally, expert witnessess are oblidged to initially presume nothing -- neither innocence nor guilt -- and approach the situation unbiasedly.

So, prosecutors assuming guilt is not contrary to the normal presumption of innocence ;)

I took it out. Figured the edit needed explainin --DakAD 15:42, 7 June 2006 (UTC)

Retrieved from "http://en.wikipedia.org../../../p/r/o/Talk%7EProsecutor%27s_fallacy_884f.html"