For over half a century, the randomized controlled trial has been the gold standard of biomedical research. Comparing how a carefully selected group of subjects responds to a new drug or therapy to the response of a comparable group given a placebo is the most rigorous way to find out if an intervention works, and if it is Safe.
But as valuable and essential as these trials are, particularly for the regulators who must decide whether to allow new interventions on the market, they have important limitations. “They take a long time and they are expensive,” says Marni Hall ’97, vice president of clinical evidence and head of U.S. regulatory science and strategy at IQVIA, which uses data, technology, and advanced analytics to help its clients advance medical research and healthcare. “And they don’t tell you what happens in the real world when a drug is on the market, when it’s used by patients who were excluded from trials due to other medical conditions or those who are taking other medications, or due to other criteria that make potential participants ineligible. The results are typically not generalizable to the public.”
Hall has spent a lot of time over the past two decades thinking about how to study the risks and benefits of medical products outside of traditional clinical trials, ways that better capture how people, broadly, actually respond to such products. She is part of a movement that advocates for employing a wider range of data in research and regulation, data that can help predict and track the performance of medications and other products in the real world, and even augment and accelerate traditional clinical trials.
The movement is focused on what is known as real-world data, which can come from a variety of sources, including electronic health records, insurance claims, product and disease registries, self-reports from patients, and even health-tracking apps on smartphones. Gathered with care and properly analyzed, real-world data can yield real-world evidence, which can play a vital role in research and public health.
Real-world evidence, Hall says, could also create a bridge between clinical trials and clinical care (treatment by health care providers). Traditionally, data from a patient’s involvement in a trial has been held apart from any other data about that patient. “But this means the process doesn’t benefit from the context and insights that may be gleaned from a more holistic view of a person’s health,” she says.
“These approaches help advance precision medicine: getting the right medicine, to the right people, at the right time,” she says, “The idea is that in our healthcare system, the current models don’t always work, for a variety of reasons, and even when they do they are not efficient. We should be able to learn more, faster, and to have patients benefit from the process. We should also have the outcomes be those that are of interest to patients while also being more generalizable.”
The need for greater use of real-world data and real-world evidence has gained increasing support in recent years, including on Capitol Hill. The 21st Century Cures Act, signed into law in 2016, and the sixth reauthorization of the Prescription Drug User Fee Act, passed in 2017, both require the Food and Drug Administration (FDA) to develop frameworks for making greater use of real-world data and evidence in regulatory decision making.
And if the real-world data community needed a case study to demonstrate, unequivocally, the value of this new way of looking at medical research and regulation, the COVID-19 pandemic has provided it. Hall, who sits on a national task force charged with developing infrastructures and methods for managing the pandemic and preparing for future disasters, says the global health emergency has shown that real-world data and evidence can help us understand a rapidly changing public health crisis and speed decisions about testing and treatments in ways that conventional research alone cannot.
Extracting Meaning
One of the principal challenges with using real-world data, Hall says, is that it can be messy and unreliable. There is also a pressing need to bring greater rigor to its collection. She spends a great deal of her time on efforts to establish standards for real-world data to assure that different data sources can be harmonized and made more interoperable.
Electronic Health Records are a case in point. “Initially, electronic health records were electronic versions of filing cabinets,” Hall says. “They were not designed to contain relational data that could be easily linked, and the systems are highly customized. So the lack of interoperability provides an important challenge toward making this kind of real-world data a reality.
In addition,” she notes, “there is a tension between the idea that we have this data and should be able to learn from it and data privacy concerns. But as the data quality improves and as better mechanisms for anonymizing integrated data are developed, use of these data will become more routine”
Hall first encountered the messiness of real-world data at the FDA, where she worked for more than seven years, first as senior program director and then as director of regulatory science in the Office of Surveillance and Epidemiology. At the FDA, she was responsible for the Adverse Event Reporting System, which collects reports from manufacturers, healthcare professionals, and patients themselves of adverse reactions to drugs—more than 15 million reports dating back to 1969, with nearly two million added each year.
“I spent a great deal of time when I was at the agency figuring out how to extract meaningful information from what is truly one of the messiest, most challenging data sets,” she says.
One of her challenges was finding innovative ways to separate signals from the noise—or put another way, to find the needles in this massive haystack. The adverse reporting system contains a combination of structured data where users choose from a limited number of options (for example, by using pull-down menus) and unstructured data (for example, a text box in which one writes a narrative describing the adverse event). Missing information, inconsistent terminology, and even bad grammar make the unstructured data almost impossible to comb through with a computer.
It’s the most powerful thing to be able to take all of this data and really put it to use.
Starting in 2015, Hall began sponsoring undergraduate MQPs (a disciplinary project all undergraduates must complete) and master’s and PhD research projects at WPI that have applied machine learning and other advanced data science tools to make sense of the written responses. The projects, co-advised by Elke Rundensteiner, professor of computer science and founding director of WPI’s Data Science Program, seek to turn those responses into searchable data that can be mined for dangerous drug interactions that might previously have been hidden.
Working with the adverse event system, Hall says, made clear not just the challenges of working with untidy data, but the importance of building data sets the right way from the start. “That work prepared me well to think about what sort of innovations you need to either create a data source, if you don’t have one, or to figure out how to use the data that you have.
“And that includes having data standards, reporting requirements, and methods and tools to deal with data as it is. That may sound boring, but it’s the most powerful thing to be able to take all this data that’s being generated and really put it to use.”
At the Intersection
While much of Hall’s work revolves around data, she is not a data scientist by training. At WPI, she completed two majors, one in chemistry and one in science, technology, and policy studies. She went on to earn a master’s in public health from Columbia University’s Mailman School of Public Health as well as a master of science in biochemistry and a PhD in toxicology from Columbia’s Graduate School of Arts and Sciences, where she has taught for several years. “I’m a bench scientist and an epidemiologist.” She says. “I don’t mess around with data. My interest is in what can be done with data.”
Her career has taken her from Pfizer, where she was program director in the Public Health Group of External Medical Affairs, to the FDA, and then to PatientsLikeMe, where, as senior vice president of research and development, informatics, and policy, she helped shape an agenda for the use of patient-generated health data in clinical and regulatory decision making. She joined IQVIA in 2018 as vice president of clinical evidence in the Center for Advanced Evidence Generation. In February 2020, just after the first cases of COVID-19 emerged in the United States, she launched a regulatory strategy team to focus on the science involved in using new data, methods, and tools where there is little or no regulatory precedent.
“The work I am doing today with COVID-19 came from my WPI training,” she says. “WPI prepares you to work in multidisciplinary teams to define and solve complicated problems, to address things that don’t go the way you expect them to, and to figure out how to contribute.
I have a lot of technical expertise, but my ability to apply that to a multidisciplinary problem—a problem that isn’t just science, but also data and policies—and to deal with the uncertainties involved, those are things the Plan prepared me well for,” she says, referring to WPI’s project-based approach to education.
A member of the WPI Board of Trustees since 2016, Hall has also served on the university’s Life Sciences Advisory Board and WIN (Women’s Impact Network). She met her husband, Edward Hallissey ’98, in a thermodynamics class at WPI; both went on to work at the FDA.
Another asset she gained at WPI, particularly through her double major, is her ability to navigate problems that sit squarely at the intersection of science and policy. “Most scientific problems today don’t live in a pure discipline,” she says. “And often, the place where multiple things intersect is where you can have the most impact. That’s not something everyone can manage, but it’s exactly what WPI trains you to do.”
An Evolving Crisis
The COVID-19 pandemic is the epitome of a multidimensional, multidisciplinary problem, one that sits at the intersection of science, technology, public health, policy, politics, and at least a half dozen other disparate disciplines. It has led Hall to draw on every bit of expertise she has gained in her distinguished career. Arriving home from an international trip in January, she was quickly caught up in a swirl of activity as IQVIA and her professional colleagues tried to keep up with a rapidly evolving and escalating public health crisis.
“We’re learning so much, and so fast, on the fly. We’re collaborating in ways that we never have before. And we’re already doing things to assess how many of these different approaches can be used in non-COVID scenarios.”
Early on, Hall joined several internal working groups. Some worked to establish COVID-19 health data codes, so scientists could accurately measure who had the disease, as well as track patient symptoms and outcomes. This made it possible to look back at data from the first quarter of 2020 to identify patients who did not receive
a COVID-19 diagnosis, even though they might have been infected by the virus. This work informed predictive models that are able to anticipate infection rates and supply chain needs.
Other working groups focused on clinical trials that clients were preparing to run or already had started. With hospitals overrun by COVID patients, doctor visits shifting to telemedicine, and research labs shuttered, the pandemic put many trials, including those for new cancer treatments, in jeopardy. “At first, there was a very quick assessment of how clinical trials were being affected and possible mitigation strategies.”
Those strategies included turning to real-world data to keep trials going. “One of the ways that’s applicable is to say, if there is a trial that has been disrupted, maybe you can use real-world data to supplement the evidence you were going to generate through clinical trials,” Hall says. “You can design a way to collect it, whether it is self-reported patient data, data from their electronic health records, or data from mobile devices. All of those things are actually happening right now in response to COVID-19.”
Learning from those activities is one of the goals of two sessions Hall will chair this year at the annual Real-World Evidence Conference sponsored by DIA, a global organization that focuses on issues at the intersection of science, health care, and regulation. She has been on the planning committee for the conference for several years. The two sessions will focus on the impact of real-world evidence on health care and research during the pandemic. Hall also regularly gives webinars to her clients, which include pharmaceutical and biotech companies, regulators, and the FDA, on real-world evidence and COVID-19.
As the winter waned and the pandemic took hold, Hall found herself confronted with a constantly shifting set of issues as a member of the national Pandemic Response Emergency Preparedness Taskforce (PREPT), a public-private partnership composed of regulators and professionals from industry and academia. The group’s charge includes scaling up COVID-19 testing to address the needs of vulnerable populations and evaluating the impact of the virus on health care delivery and patient outcomes. One of the early challenges the task force took on was the supply chain.
“In some ways,” she says, “that is a data science problem—just figuring out where things are. At first it was personal protective equipment and ventilators, then we turned to reagents for tests, and then the pipette tips to deliver the reagents. The standard protocol for one test requires 19 pipette tips. One indication of how fast-moving this crisis has been is that the things that consumed us a few months ago are totally different than the things that consume us today.
The acceleration of learning made possible by the sharing of new approaches to using real-world data is one outcome of the COVID crisis that Hall says she hopes to see continue beyond the crisis. Another is the collaboration of multiple stakeholders and multiple interests toward reaching a common public health goal. “There is so much that is tragic about COVID-19,” she says, “and this crisis has taken people away from other important activities, but it is also hopeful, in a way. It may prove to be a force function or an accelerant to advancing precision medicine and the use of real-world evidence, and it may teach us how to collaborate in new and different ways that will ultimately improve public health. To me, that is encouraging.” [ J ]