Academic Commons

Theses Doctoral

¿When is it the Gesture that Counts: Telling Stories that cut to the [Cyber]chase – or, gest get to the po¡nt!

Swart, Michael Isaac

Lakoff and Nuñez (2000) argue that the origins of mathematical thinking arise from the progressive development of the human sensorium and experience. Cognitive science research in in education plays a big role in developing new pedagogies, especially those that leverage new “Cyberlearning” technologies. The current study employs two principle frameworks for creating pedagogy for learning mathematical fractions: (1) grounded and embodied cognition (Varela, Thompson & Rosch, 1991; Glenberg, 1997; 2003; Barsalou, 1999; 2008), (2) situated cognition (Lesh, 1981; Lave 1988, Greeno, 1998; Roth, 2002). Grounded and embodied cognition was operationalized through the gesture. Although gesture is traditionally discussed as a spontaneous co-articulation of speech (Kendon, 1972; McNeill & Levy, 1980; 1992; Goldin-Meadow, 1986) it is taking on a new role with the advent of 21st century technologies that utilize gestural interface. Using gestures as simulated action (Hostetter and Alibali, 2008), we developed two sets of gestural mechanics based on an exploratory study on the gestures elementary students used to explain mathematical fractions (Swart, 2014): (1) iconic gestures (I) – i.e., enactive of the processes to create objects, (2) deictic gestures (D) – i.e., index pointing to ground or identify objects or locations.
Situated cognition was operationalized through narrative (Black and Bower, 1980; Graesser, Hauft-Smith, Cohen, and Pyles 1980; Graesser, Singer, Trabasso, 1994). Researchers crafted two types of narratives in order to create a situated learning environment (Hennessy, 1993): (1) strong narrative (S) – with a setting, characters and plot (based on the popular PBS Kids television show, Cyberchase, (2) weak narrative (W) – without an explicit setting, characters or plot. Combining these two factors together, the research team designed and developed Mobile Mathematics Movement (M3). Using the two independent variables, gesture (I vs. D) and narrative (S vs. W), M3 was crafted into 4 different versions: SI, SD, WI, WD. The first two iterations, M3:i1 and M3:i2, were tested in randomized factorial experiments in afterschool programs with high-needs populations. After completing these studies employing a design-based research (DBR) methodology, the tutor-game developed into its latest iteration, M3:i3. The curriculum of M3 had students employing a splitting objects (i.e., parts-to-whole) schema (Steffe, 2004) and was divided into two parts: (Part 1) object fracturing (x5 per level): estimating, denominating, numerating, re-estimating; (Part 2) object equivalency (comparing 5 fractions): comparing, ordering, verifying magnitudes, verifying positions on vertical number line.
In the final dissertation study, 131 students (x̄age = 8.78 yrs, 52.6% Female; 39.7% Hispanic; 32.8% African-American; 19.9% South-East Asian; 3.8% Caucasian; 3.8% South Asian (Indian); 97.7 % received free/reduced lunch) from the Harlem and Lower East Side neighborhoods of New York City were consented and assented and completed the study. Students were randomly assigned to 1 of the 4 conditions, completed a direct pre-assessment of the curriculum as well as a transfer pre-assessment, played all seven levels of the tutor-game, completed an exit survey (free response and 5-point likert – motivation, self-efficacy, engagement, learning), completed a direct post-assessment of the curriculum as well as a transfer post-assessment (parallel forms) and a 7 minute semi-structured clinical interview.
Factorial ANOVAs indicated a significant interaction between gesture and narrative (though all groups showed significant learning pre to post) on the direct assessment. Both the SI and WD groups significantly outperformed the other two groups, though were not different from each other. Though there was not a significant interaction between gesture and narrative on for the transfer assessment, pair-wise comparisons and planned contrasts showed that the SI group outperformed all the other groups. Follow up hierarchical linear regressions (HLR) showed that game play significantly mediated students’ learning. Specifically, students’ performances estimating and denominating were predictive of direct learning of the curriculum while estimating, denomination and numeration were all predictive of transfer. Further HLRs also found that students’ learning was moderated by their existing proficiencies for fractions. This finding helped clarify the nature of the narrative-gesture interaction, such that low-proficiency students improved more in the WD condition and high-proficiency students improved more in the SI condition. An exploratory factor analysis of the 5-point likert exit survey showed loaded on four factors as anticipated, with significant loadings for engagement and learning, but revealed no significant differences between the conditions.
The significant interaction revealed that both a weak narrative (non-contextualized) environments using deictic (identity) gestures as well as strong narrative (contextualized) environments using iconic (enactive) gestures are differentially beneficial for learning. Contrary to our interaction hypothesis, learning for novices benefitted from a more abstract environment, supporting the work of (Kaminski, Sloutsky, Heckler, 2008) and learning for those with higher proficiencies at fractions was better in the more concrete environment (e.g., Moreno, Ozogul, & Reisslein (2011). The likert data supports research suggesting that students find digital platforms engaging and empowering, regardless of learning or not (for review see Wouters, van Nimwegen, van Oostendorp, & van der Spek, 2013). Together, these results have important implications for the design of learning environments and a digital pedagogy and follow-up work is necessary for expounding on the interactions between gestures and narratives as well as the possible mediation by task complexity.


  • thumnail for Swart_columbia_0054D_13641.pdf Swart_columbia_0054D_13641.pdf binary/octet-stream 93.7 MB Download File

More About This Work

Academic Units
Cognitive Studies in Education
Thesis Advisors
Black, John B.
Ph.D., Columbia University
Published Here
October 19, 2016
Academic Commons provides global access to research and scholarship produced at Columbia University, Barnard College, Teachers College, Union Theological Seminary and Jewish Theological Seminary. Academic Commons is managed by the Columbia University Libraries.