Blue Pill or Red Pill?
Down the Rabbit Hole of Comparative Effectiveness Research
With these words, President Barack Obama not only demonstrated his hip sci-fi credentials—Morpheus’s choice to Neo was either to take the blue pill and remain happy but ignorant of the truth, or the red pill, which would reveal to him a sometimes-painful reality and also launch the lucrative “Matrix” trilogy of movies—but also his desire to take a 21st-century, data-driven approach to clinical decision making and health care policy.
Among competing treatments for the same disease, which one is best? Which one is worth the money? These questions are the core of comparative effectiveness research. Half of insured patients in the United States are on chronic medications for conditions such as diabetes, hypertension, and high cholesterol. Patients, physicians, and policymakers need reliable data to know what to take, what to recommend, and what is worth paying for. Typically, however, they don’t have these data.
The Affordable Care Act, better known as Obamacare, has implemented a number of initiatives to address this problem. One of the largest is the Patient-Centered Outcomes Research Institute, or PCORI. A core mission of PCORI is to conduct comparative effectiveness research that gives patients and their health care providers the best evidence to help make more informed decisions. As promising and common sense as this mission is—because why not pay half price?—solid gold evidence to answer a patient’s question “Should I take the red pill or blue pill?” is hard to obtain.
The fundamental problem is that the gold standard for studying comparative effectiveness, the randomized controlled trial, or RCT, is too costly and disruptive to be done for every important comparative effectiveness question. At the RCT’s core is the assignment of an intervention to each subject by a “flip of a coin,” meaning that some patients receive drug A, and some patients receive drug B.
Unfortunately, an RCT is a massive enterprise. Special procedures such as using a “flip of a coin” at a central study site to assign each patient to an intervention are so different from routine clinical practice that trials must hire expert clinical investigators and take place at special study sites. Meanwhile, patients and physicians alike can be reluctant to engage in an activity so potentially disruptive to routine clinical care. Ethical oversight helps ensure that clinical care is not truly compromised, but this oversight is intensive, costly, and time-consuming too. The result? RCTs can take years and cost millions of dollars.
Given these challenges, relatively few RCTs are done. The major pharmaceutical companies are among the few institutions that can single-handedly muster the resources to implement large RCTs, and they use them to get their drugs approved, typically by comparing the drug to a placebo. Even if multiple competing drugs are available for a disease, drug companies rarely conduct comparative effectiveness studies. Why should they? For a company, the decision to conduct a RCT is not a matter of public policy. It’s a business decision. If a company does a comparativeness effectiveness trial, the study design often uses clever design features that, unsurprisingly, stack the data to show their that drug is more effective.
Fundamental questions—such as “Does drug A or drug B have a better chance at keeping a diabetic patient from needing insulin? Does drug A or drug B prolong life more in heart failure patients?”—go unanswered because, outside of the big pharmaceutical companies, few institutions have the resources to do an RCT to answer these questions. Part of the answer may be for the Food and Drug Administration to ask for more RCTs to address comparative effectiveness questions, but we also need new methods to do comparative effectiveness research more efficiently.
The RCT is a 20th century method that worked well for acute, serious diseases such as infectious diseases, heart attacks, and pediatric cancers, where entry criteria were simple, options in clinical care were few, and results could be obtained relatively quickly. Since the middle of the 20th century when the debt-weary post-war British National Health Service used it to inform whether streptomycin therapy was worth the cost for the treatment of tuberculosis, it has been the court system that decides which promising therapies are in fact safe and effective and which are not. For complex, common, and chronic diseases such as diabetes that can require lifelong treatment, however, the RCT is a large and costly enterprise akin to moving an armada across an ocean.
President Obama’s call for a trial to compare the blue pill to the red pill would mean mustering millions of dollars and recruiting thousands of patients as research subjects to be followed for many years. Even then, the results will likely be subject to a fusillade of questions because patients who participate in an RCT are typically not like the usual patient, the protocols often limit usual care, and treatment options may have changed in the years it took to execute the trial. A more modern, streamlined approach is needed.
Just as the RCT was made possible by 20th-century advances in statistics and research technologies, 21st-century advances now present an alternative to the large, expensive, and cumbersome clinical trial. The critical change happening now is the linking of fast, user friendly, networked computers into large databases replete with medical information—the so-called electronic medical record, or EMR.
Most proposals to use EMR as a tool for comparative effectiveness research simply use the EMR as a large database for a traditional observational study. This possibility has received deserved attention, but has also been appropriately criticized, because such traditional observational studies are not nearly as reliable as RCTs in distinguishing true causal effects of drugs from non-causal associations. We propose a complementary way to use EMR that will retain some of the special advantages of RCTs at much lower cost and with fewer ethical problems. We call it Prompted Optional Randomization Trial, or PORTS, a design impossible in the days of paper charts but easily implemented [subscription required] using an EMR.
Physicians who use the EMR have experienced how the system talks back to them. It can, for example, prompt a physician to reconsider or even change a medication that is linked to a documented patient allergy, interacts with another medication, or is not on formulary. These prompts sometimes result in rapid, appropriate adjustment of medications, but perhaps more often the physician finds the suggested change inappropriate and overrides the prompt with the click of a button.
The same technology can be used to introduce one of the RCT’s essential features, the “flip of a coin,” where the computer can choose whether the patient receives the red pill or the blue pill. Whenever a physician order one of these colorful pills, the computer can make its own random choice between the drugs. The computer can then prompt a physician to consider changing his or her prescription, but only when a physician’s order and the computer’s random choice are discordant. When identical to the randomly generated orders,, physicians’ orders stand.
If, for example, a physician orders the blue pill, 50 percent of the time the computer will also choose the blue pill. No prompt will be displayed, and the physician prescribes blue. If the computer chooses red instead, it displays a prompt to consider prescribing the red pill instead of the blue pill. A physician who prefers the blue pill for the particular patient dismisses the prompt with a single click and prescribes the blue pill. A physician with no preference between treatments, however, can endorse the change with a single click and prescribe the red pill.
A PORTS study design makes sense when the red pill and the blue pill are both used interchangeably in clinical practice, but physicians truly do not know which one is safer or more effective. This design increases the probability that a patient will receive the randomly assigned treatment. The association will not be perfect, since in many cases the patient and physician will prefer a drug and appropriately ignore a prompt that conflicts with that preference. Intuitively, however, if the blue pill is in fact a little better than the red and a prompt for blue makes patients more likely to get blue, the patients who do get a prompt for blue will on average do a little better than patients who get a prompt for red.
Crucially, that difference will reflect the properties of the pills themselves, not subtle differences between the kinds of patients who choose red and those who would rather have blue. A relatively simple technique called instrumental variable analysis formalizes this intuition and makes it possible to take these data and uncover the difference in effectiveness between the red pill and the blue pill. It turns routine clinical practice into an efficient and low-cost engine of discovery that will tell Americans whether we should take the red pill or the blue pill.
To be sure, this method has ethical challenges. Some patients will get a treatment different from what they and their doctor would have otherwise selected. Is it possible that some patients will be harmed? We would argue that it is not, because the physician can override the prompt if there is any reason to suspect one drug is worse than the other. Should patients give consent before this method is used? Would they need to give it every time, or just once when they establish care at a practice that uses this method? These are questions that need to be addressed, but they are mere shadows compared to the glare of the serious ethical concerns traditional RCTs raise.
To date, the EMR has received middling marks as a technology to reduce health care costs. The PORTS proposal is just one example of the more general but untapped promise of the EMR in medicine, a promise that could be as revolutionary as the RCT, and before that, the stethoscope.
Electronic systems, prompts, and other tools can introduce small probabilistic changes in care, changes that can yield the kind of unbiased quality improvement data that to date has been available only at the high cost of the RCT. Small, benign random variations in practice could gradually develop a far more comprehensive picture of what works and what does not.
We just have to summon the will to take the red pill and discover the innovative ways to interact with the new matrix of medical data.
James Flory is a fellow in endocrinology at Weill Cornell Medical Center. Jason Karlawish is a professor of medicine, medical ethics and health policy at the University of Pennsylvania. Image: Warner Bros / Village Roadshow Pictures.
Comments on this article