2025 Theses Doctoral

Leveraging Large Language Models to Enable Drug Safety Research

Gisladottir, Undina

Adverse drug reactions, including those resulting from drug interactions, remain a leading cause of morbidity and mortality. In the United States, 82% of adults take at least one medication and nearly a third of the population takes five or more. The post-market surveillance of drugs is crucial to ensure the ongoing safety and efficacy of medical treatments, helping to identify potential risks that may not be evident in pre-market trials. Structured product labels (SPLs) serve as a primary source for drug safety information. Having readily available machine-readable product labels, including adverse drug reactions (ADRs) and drug interactions, would allow researchers to streamline medication safety studies.

However, extracting this information is complex and requires the use of natural language processing (NLP) methods. Once extracted, this information is useful for understanding what is currently known about the drug. However, the drug safety information stored in SPLs does not aid in high-throughput research or the evaluation of drug safety research methods for two reasons. First, this information is not stored in a machine-readable format and second, the adverse reactions listed in SPLs are not all causally associated with the drug.

In Chapter 3, I address the need for a machine-readable database containing vital drug safety information from SPLs. Previous efforts have relied on BERT-based models in the extraction of adverse reaction terms from SPLs. However, these models require extensive training for each specific extraction task. In this chapter, I explore the application of generative language models in the extraction of drug safety information from SPLs. Specifically, I use these models to extract adverse reaction terms from relevant sections of the SPL and compare the performance to baseline models.

Additionally, I apply the best-performing model to extract drugs from the drug interaction section without any additional training. Here I show that generative models, specifically GPT-4, are able to match the performance of the state-of-the-art BERT-based models without the need for manually created training data. Additionally, I demonstrate the broader applicability and flexibility of these generative models by applying them to previously unannotated sections.

While accurately extracted adverse reactions from SPLs are helpful for future research, they cannot be used as positive controls in drug safety research because many of these reactions are likely not causal. In Chapter 4, I develop a method to identify the subset of reactions likely to have a causal association with the drug from all SPL-listed reactions. This work addresses the need for an automated method for the development of large-scale reference standards. Up-to-date and broad reference standards, which classify adverse drug reactions as positive or negative controls, are essential to allow for the evaluation of methods and serve as comparison groups for novel signal detection. The development of the current gold standard of reference standard by Ryan et al. required the extensive efforts of drug safety experts.

Due to the intensive process, which included a thorough literature review, this reference standard only spans four adverse reactions. As the majority of adverse reactions listed in SPLs are not associated with drug exposure, I begin this chapter by investigating whether the leading generative language model, GPT-4, can determine the causality of SPL-listed reactions. I then explore the application of generative language models in summarizing evidence of a causal relationship between adverse reactions and drug exposures in place of extensive literature reviews. Using different classifiers and established reference standards, I developed a method to identify positive controls from all SPL-listed reactions. This work demonstrates the application of generative language models in the detection of positive controls for drug safety research.

Together, the studies in this dissertation work to make essential drug safety information readily available for high-throughput research and the evaluation of research methods. In Chapter 5, I contextualize this work and provide the conclusions, limitations, and implications of the research findings as well as recommendations for future work.