David Gorski in "“Moneyball,” the 2012 election, and science- and evidence-based medicine" has yet another post that brilliantly teaches us mere mortals something useful about medicine, moneyball, political predictions, and the Bayes statistical methods now famously used by Nate Silver to make election predictions based on prior knowledge about the electorate:
"Critics of moneyball approaches have nonetheless been quick to emphasize the way in which perspective can be distorted, not enhanced, by statistics. One might overapply concepts such as Bayes’ theorem or develop a habit of plugging data into statistical software simply to gain a patina of precision, regardless of appropriateness (tendencies that cause medical practitioners, in Alvan Feinstein’s pithy phrase, to be blinded by the “haze of Bayes”). Critics have also pointed to what might be termed the “uncertainty principle” of statistical analysis: general data (How well does this player hit against left-handers? How well does this therapy work in myocardial infarction?) often fail to take into account consequential distinctions; but more specific data (How well does this player hit against hard-throwing left-handers on warm Sunday afternoons in late September? How well does this therapy work in right-sided myocardial infarction in postmenopausal women?) can involve too few cases to be broadly useful. Individuals, and individual scenarios, might always be idiosyncratic on some level — a truth perhaps borne out by long-standing efforts to appropriately apply the scientific results of clinical trials to individual patients in the clinic."
"Here’s where Phillips et al go wrong. As we have pointed out here on this very blog many, many times before, the problem with Evidence Based Medicine (EBM) is not that it overapplies Bayes’ theorem. Rather it’s that EBM uses frequentist statistics over Bayes’, often massively underplaying the importance of prior plausibility, estimates of which Bayesian statistics demands. Indeed, not making Baysian estimates of prior plausibility leads to EBM’s blind spot towards Complementary and Alternative Medicine (CAM), such that statistical noise and bias in trial design can lead to the appearance of efficacy for therapies whose rationale, to be true, would require that much of modern physics and chemistry be not just wrong but massively wrong. I’m talking, of course, about modalities like homeopathy, reiki and other “energy healing” treatments, and the like. While “moneyball” does take Bayes into account somewhat, there is one big difference between moneyball and medicine, and that’s science. There are certain things in medicine that can be dismissed as so implausible that for all practical purposes they are impossible based on physical laws and well-established science (i.e., homeopathy). The result was hilariously illustrated in a very simple way in a recent XKCD cartoon that asked if the sun just exploded."
@ xkcd: Frequentists vs. Bayesians:
Gorski -- the man is brilliant. Words just flow from the pen, every week in coherent literate witty streams. The Pundit is green with envy.
No comments:
Post a Comment