2026 Theses Doctoral

Investigating Situated Specific Purpose Writing Ability in a Scenario-based Language Assessment of Science: With Focus on the Role of Domain Knowledge

Joo, Soo Hyoung

The current study aims to investigate the construct of situated specific purpose writing ability in a scenario-based language assessment (SBLA) for science purposes, with a focus on the role of domain knowledge (domK). Using the learning-oriented language assessment (LOLA) as a guiding framework for the design of the SBLA and formulation of validation claims, this study explored how different types of domK (i.e., operationalized by means of domain-general knowledge and domain-specific knowledge) and situated specific purpose writing ability can be measured in the context of performing a complex disciplinary practice of science, how they are interrelated, and whether examinees can develop them through the assessment experience.

The SBLA for science purposes (SBLA-SP), designed for this study, features a scenario narrative modeling the habits of mind required for one type of science disciplinary practice defined by the Next Generation of Science Standards (NGSS) framework: planning and carrying out investigations. To fulfill the scenario goal, examinees were guided through a series of tasks intended to progressively build their domK, culminating in the task of writing a lab report. Domk items aligned with the NGSS were delivered both before and during the scenario to track the development of domK.

The findings indicated that the SBLA-SP provides a valid and reliable means of eliciting and measuring both domK and situated specific-purpose writing ability, while also supporting meaningful domK development across the scenario, particularly in dom-specK alongside more consistent growth in dom-genK. The results further demonstrated that both dom-genK and dom-specK are integral to writing performance, with dom-genK facilitating the activation of dom-specK during task engagement, and that these relationships vary by proficiency, with the most stable patterns observed among intermediate learners. Together, these findings demonstrate that SBLA can effectively simulate disciplinary practices in STEM, thereby informing the design and validation of language assessments that more authentically elicit and measure the competencies required for real-world disciplinary performance.