GMO Statistics Part 28. The full story really matters for making a sound judgment about what a test result means

An earlier post presented mammogram test results on women with breast cancer as follows, where positive tests results are shown in yellow, and the definite presence of cancer in 1000 adult women is shown as a red dot. 

(This is a way of presenting scenarios with mammagrams used by Nate Silver in the now best selling book The Signal and the Noise.

The panel above presents tests only done on all women who really do have have cancer. It does not reveal what test results are found on women known NOT to have breast cancer.

The testing shows 11 of the women with breast cancer had a positive mammogram  and 3 women were missed by the mammogram test. This absence of yellow background behind the red dots is called a false-negative result, because even though the mammogram test was negative, the unfortunate women still had breast cancer.

In that post we posed this question to readers:

This brings me to a question I have for any women reading this blog posting.
If you had a positive mammogram result given to you, and didn't know your breast cancer status, how would you react if you were told your test was positive? What would you believe to be the odds that you don't have breast cancer even though the test was positive? Would you be dreadfully depressed? Does the above panel help you in coming to a decision? Do you have a better way to examine this issue?

One possible response is for a women to see the numbers of positive results compared to negative results, in the ration 11 is to 3, and ton feel that its highly certain a positive test result indicates that they have cancer.

But there is vital information missing from the diagram above . This is the outcomes of mammogram results seen on women who do NOT have breast cancer:

There 986 women who are free from breast cancer of the thousand in the entire population. There are many more positive mammogram results (99 yellow squares) among these women than seen with the women who do have breast cancer (red dots). 

By considering this extra information, the significance of a positive mammogram to any randomly chosen women who knows nothing else about her breast cancer status might seem to be different than if the were told the test doesn't miss many cancers.

The most encouraging news for women taking the test is the sea of yellow in the diagram. Among the yellow squares on the figure, red dots on yellow squares representing detected cancer are relatively rare. This means that for a women who is tested by a mammogram in a random screening program, the good news is that  a positive test result means the odds she doesn't have breast cancer are still 99:11. Most likely she is cancer free.

Why this, at first glance odd result: that a positive test result means that you are most likely free of cancer?

It lies with the low prior probability that a women in the general population actually has breast cancer. In this discussion this low probability is 14 per 1000. 

Mammogram screening of all the 10000 women gives mostly false-positive results (887/1000). Thus with this rare disease (in young populations of women), false positive results dominate the screening test outcomes. For careful and helpful  understanding of the prospects of disease in any woman after her mammogram test result becomes available (the prospects of real disease or the prospective-predictive value of the test), the prior-probability of the disease (before the test) is crucial background knowledge.

Women (and particularly younger women) need to know and be told that breast cancer is rare (and improbable), and that most mammogram positive tests in younger women are false-alerts.

See also

R. Feynmann. Some remarks on science, pseudoscience, and on learning not to fool yourself. Caltech's 1974 Commencement address (pdf)