A false sense of certainty – Games
Knowledge, playfully
A false sense of certainty
A test is positive – so am I ill? Not necessarily.
A test result feels like certainty: “positive” seems unambiguous, “negative” too. But how much a result really means often depends less on the test itself than on how common a condition is in the group being examined. That leads to a result which surprises many people.
An example to think through
Imagine a disease that 5 in 1000 people have – so it is rare. The test is good and correctly identifies almost all ill and almost all healthy people. And yet: among the many hundreds of healthy people, the test wrongly classifies a few as “ill”. But because there are so many healthy people, this adds up to more false alarms than genuine cases. Of everyone who receives a positive result, only a small part is really ill – so a positive test here is a clue, but far from a verdict.
Conversely, with a well-founded suspicion, a negative result can be misleading – genuine cases are then missed. An example closer to our everyday practice:
A second example – from practice
You twisted your knee during sport and suspect a meniscus tear. Because the symptoms fit well, the probability is high from the outset – let us assume that of 1000 people with such a suspicion, about half really have a tear. The doctor or therapist examines the knee with the McMurray test, a widely used manoeuvre.
Studies show: this test detects a genuine tear in only a good 6 out of 10 cases, but rarely gives a false alarm in healthy people. The consequence – a positive result argues fairly clearly for a tear (about 8 out of 10 are correct). A negative result, however, is no all-clear: almost 200 out of 1000 with a genuine tear are missed, and about a third of those with an unremarkable test still have a tear. With persistent symptoms it is therefore worth taking a second look – for example another test or imaging.
Test accuracy according to a systematic review: McMurray test around 61 % sensitivity and 84 % specificity. The assumed pre-test probability serves illustration. You can try this case directly below as the “Meniscus” preset. doi:10.1136/ebmed-2014-110160
The same result – once positive, once negative – therefore means something quite different depending on how common the condition is.
The rarer something is, the more cautiously you take a positive result. The more common something is, the more cautiously a negative one.
The four terms in brief
- Prevalence (how common)
- How many of 1000 people really have the condition.
- Sensitivity
- How well the test detects ill people – the higher, the fewer are missed.
- Specificity
- How well the test detects healthy people – the higher, the fewer false alarms.
- False alarm (false positive)
- The test says “ill”, but the person is healthy.
- Missed (false negative)
- The test says “healthy”, but the person is ill.
Try it yourself now: move the sliders below and watch how the block of 1000 people rearranges itself.
Adjustments
How many of 1000 people actually have the condition?
Share of ill people the test correctly reports as positive.
Share of healthy people the test correctly reports as negative.
1000 people
Test positive → really ill?
Test negative → really healthy?
Sources
- Gigerenzer G, Gaissmaier W, Kurz-Milcke E, Schwartz LM, Woloshin S. Helping Doctors and Patients Make Sense of Health Statistics. Psychological Science in the Public Interest. 2007;8(2):53–96.
https://doi.org/10.1111/j.1539-6053.2008.00033.x - Smith BE, Thacker D, Crewesmith A, Hall M. Special tests for assessing meniscal tears within the knee: a systematic review and meta-analysis. Evidence-Based Medicine. 2015;20(3):88–97.
https://doi.org/10.1136/ebmed-2014-110160 - Hegedus EJ, Cook C, Hasselblad V, Goode A, McCrory DC. Physical examination tests for assessing a torn meniscus in the knee: a systematic review with meta-analysis. Journal of Orthopaedic & Sports Physical Therapy. 2007;37(9):541–550.
https://doi.org/10.2519/jospt.2007.2560 - Brinjikji W, Luetmer PH, Comstock B, et al. Systematic Literature Review of Imaging Features of Spinal Degeneration in Asymptomatic Populations. American Journal of Neuroradiology. 2015;36(4):811–816.
https://doi.org/10.3174/ajnr.A4173
A simplified illustration – not medical advice. “Prevalence” here means how common the condition is in the group examined; “sensitivity” and “specificity” describe how reliably the test identifies ill and healthy people.