A man fires a gun several times at the side of a barn and then draws a circle around a cluster of most of the bullet holes. Drawing a target retrospectively like this doesn't prove the shooting skills of the gunman – no one would consider him a sharpshooter if they knew what he'd done. When the equivalent of this happens in other circumstances we call it the Texas sharpshooter fallacy. As with many fallacies, it may not appear fallacious at first inspection.
The classic example of this fallacy is in the identification of a cancer cluster: a geographic area with a perceived or real above average number of cancer cases thought to be in need of explanation. Typically something in the local environment is claimed to be the cause – e.g. overhead power lines, pollution from a chemical plant etc. The problem is that the geographic area chosen is akin to the shooter's circle drawn around bullet holes after firing the gun.
If the cancer cluster is large enough and occurs in a small town then the localised cancer rate is likely to be higher than the national average. However, if the geographic area selected was a county, then the rate is likely to be close to the national average, and the cluster isn't necessarily in need of investigation. This isn't to say that cancer clusters are never the result of a localised cause, just that the existence of a cluster isn't sufficient to infer it. Public concern has led to many cancer clusters being investigated, often fruitlessly.
To understand why a cluster of something isn't necessarily in need of investigation we look at randomness. People intuitively, but wrongly, tend to think that the law of large numbers applies to small numbers – that is, they believe a small sample will be as representative of a population as a large sample. When asked to generate a hypothetical random sequence of coin tosses, people usually produce short sequences with the ratio of heads to tails close to 50 per cent; however, this is not what chance would predict. In other words, even though tails (T) in the sequence TTTTTTH occurs 6 out of 7 times consecutively it's not a reason to suspect cheating.
An example from pseudoscience is the so-called Bible Code. Here, an algorithm is chosen to arrange the biblical text in a certain way, then messages are deciphered from the result. These supposed messages are only found after playing around with the algorithm. To expose the Bible Code as an instance of the fallacy it's been shown that ‘messages’ can be found in books without the prophetic overtones of the Bible – e.g. Leo Tolstoy's novel War and Peace.
In science, a hypothesis is made then data is collected and analysed to see if it conforms to what is expected if the hypothesis is true. The fallacy occurs when a hypothesis is constructed and tested on the same data. To use the shooting analogy: a target should first be drawn, shots fired, and then the result analysed.
People who are quick to judge a situation sometimes use an old saying that there's no smoke without fire. The Texas sharpshooter fallacy reminds us that there may not actually be any smoke, let alone a fire.
