1 Real World Data, Guidelines and Decision SupportMartin S. Kohn, MD, MS, FACEP, FACPE Chief Medical Scientist, Sentrian Real World Data, Guidelines and Decision Support
2 Limitations of Traditional GuidelinesConsensus documents rather than truly evidence based Data usually from published trials Inherently short term Artificial environment Population level comparisons Focused on one disease state Null hypothesis constraint May be out of date by the time they are published; not easily amended
3 Published Literature It is simply no longer possible to believe much of the clinical research that is published, or to rely on the judgment of trusted physicians or authoritative medical guidelines. Marcia Angell, former editor-in-chief, NEJM Drug Companies & Doctors: A Story of Corruption NY Review of Books Jan 15, 2009 “A lot of what is published is incorrect.” ... much of the scientific literature, perhaps half, may simply be untrue. Richard Horton, Editor-in-Chief The Lancet Vol 385 April 11, 2015
4 Published Data - John Ioannidis“Much of what medical researchers conclude in their studies is misleading, exaggerated, or flat-out wrong.” and-medical-science/308269/, accessed 3/11/2014 “There is increasing concern that in modern research, false findings may be the majority or even the vast majority of published research claims.” PLoS Med. Aug 2005; 2(8): e124. Published online Aug 30, doi: /journal.pmed
5 Pre-Clinical Pharma ResearchScientific findings were confirmed in only 6 of 53 (11%) “landmark” papers. Begley CB, Ellis LM. Raise Standard for Preclinical Cancer Research. Nature 29 Mar : …general impression that many results that are published are hard to reproduce. …only in ~20–25% of the projects were the relevant published data completely in line with our in-house findings Prinz F, Schlange T, Asadullah K.NATURE REVIEWS | DRUG DISCOVERY 2011
6 Triple Blinding One investigator — or, more typically, a suitable computer program — methodically perturbs data values, data labels or both, often with several alternative versions of perturbation. The rest of the team then conducts as much analysis as possible ‘in the dark’. Before unblinding, investigators should agree that they are sufficiently confident of their analysis to publish whatever the result turns out to be, without further rounds of debugging or rethinking. MacCoun R, Perlmutter S. Hide results to seek the truth. Nature Oct 8, 2015
7 July 4, 1968
8 Big Data and Learning Health SystemsNew Data Sources New Thinking Inductive Reasoning Detecting Patterns Learn from Daily Experience Many Dimensions of Data The Way Massive Data Sets are Used Krumholz HM. Big Data and New Knowledge in Medicine: The Thinking, Training and Tools Needed for a Learning Health System. Health Affairs. 33:7: July 2014
9 Uncertainty IBM Global Technology Outlook 2012
10 Future of Clinical Decision SupportWe will have massive data streams resulting from pervasive monitoring and interactions with personal health monitors, the environment, and related public health data…the genome, metabolome, proteome, and microbiome. This implies the very nature of knowledge, and reasoning or decision-making, are changing under our feet. Middleton et al. Clinical Decision Support: a 25 Year Retrospective and a 25 Year Vision. IMIA Yearbook of Medical Informatics 2016
11 Cycle of evidence in rapid-learning health careAbernethy A, et al. J Clin Oncol. 2010;28:
12 The algorithms of machine learning, which can sift through vast numbers of variables looking for combinations that reliably predict outcomes, will improve prognosis, displace much of the work of radiologists and anatomical pathologists, and improve diagnostic accuracy Predicting the Future — Big Data, Machine Learning, and Clinical Medicine. Ziad Obermeyer, M.D., and Ezekiel J. Emanuel, M.D., Ph.D. N Engl J Med 2016; 375: September 29, 2016DOI: /NEJMp
13 “For an idea that does not at first seem insane, there is no hope.”Albert Einstein
14 Questions? [email protected] [email protected]