2026 Theses Master's

Assessing a Large Language Model as a Genetics Referral Support Tool Among Primary Care Providers: a Feasibility Study

Larson, Elizabeth; Morales Corado, Andres; Ahimaz, Priyanka Rebecca; Berger, Sara

Objective: To compare the use of an online educational module, a large language model (LLM) with a prompt manual, and no intervention for primary care providers (PCPs) to accurately identify appropriate pediatric indications for referral to clinical genetics.

Methods: A cross-sectional study was conducted using an online survey including ten pediatric case vignettes. Pediatricians and family medicine providers were recruited and randomly assigned to one of three groups: educational module (n=14), LLM with prompt manual (n=14), or no intervention (n=12). Participants determined whether each case warranted referral to genetics or not. Referral accuracy was compared across groups using Kruskal Wallis and chi squared analysis with α set at <0.05.

Results: Forty participants were included in the analysis. Mean referral accuracy was highest in the LLM group (9; SD=1.04), followed by the educational module group (7.79; SD=1.19) and the no intervention group (7.08; SD=1.38). Differences across groups were significant (χ²(2)=12.11, p=0.002). The LLM group outperformed both the educational module (p=0.012) and no intervention groups (p=0.0015), while no difference was observed between the educational module and no intervention groups (p=0.161). Case-level performance varied, with the greatest difference observed in scenarios involving a neurological disorder (p=0.0044) and a disorder of sexual development (p=0.0091).

Conclusions: The LLM group’s scores were associated with better referral accuracy compared to groups with online educational approaches or no intervention. These findings support the potential role of LLMs as tools to enhance clinical decision-making and improve accuracy for pediatric PCP referrals to genetics. Case specific analysis identified potential areas of gaps in genetics education in pediatric providers, including neurological disorders and disorders of sexual development. Further research is needed to explore clinical implementation in larger cohorts.

What’s New: For pediatric PCP’s, structured LLM support provided the most accurate guidance on appropriateness of genetics referrals, outperforming the online educational module group and the control group. This suggests the utilization of LLMs as a strategy to improve genetics referral decisions in pediatric care and identify potential gaps in provider education.

Keywords: pediatrics, family medicine, provider education, ChatGPT, genetics